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The present invention relates to an image 



processing method and image processor for creating 
image data with an digital watermark embedded therein 
and/or extracting an digital watermark portion from the 



In recent years, due to explosive development and 
widespread of computers and their networks, a variety 
of information such as character data, image data and 

15 voice data have been digitized. Digital information 

does not undergo degradation due to secular changes and 
thus can be stored in perfect state permanently, but 
it can be easily duplicated and therefore the 
protection of copyright becomes an important issue. 

20 For this reason, security technologies for protecting 
copyright assume greater importance at a rapid pace. 

One of techniques for protecting copyright is an 
"digital watermark". The digital watermark is a 
technique of embedding the name of a possessor of 

25 copyright and the ID of a purchaser in digital image 

data, voice data, character data and the like in a form 



image data, and a storage medium. 
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imperceptible by persons, and tracking unauthorized use 
thereof by illegal copying. 

The digital watermark may be subjected to variety 
of attacks. They include, for example, irreversible 
5 compression such as JPEG, geometric transformation such 
as scaling or rotation, printout /scanning and noise 
attachment . 

Here, a process called registration may be used 
particularly for providing resistance to geometric 

10 transformation. Registration is a process of adding a 
specific signal (registration signal) to an image in 
addition to added information at the time of embedding 
the digital watermark, and promoting the extraction of 
added information using the above described 

15 registration signal before added information is 

extracted, at the time of extracting added information. 

Conventionally, the registration signal having a 
symmetric axis is embedded when an digital watermark is 
embedded, whereby the image with the digital watermark 

20 embedded therein can be subjected to frequency 

conversion to analyze the scaling and rotation applied 
to the image. 

However, when rotation applied to the image is 
analyzed, the direction of the rotation applied to the 

25 image with the digital watermark embedded therein 

cannot be uniquely determined due to the symmetry in 
the amplitude spectrum of the spatial frequency domain. 




SUMMARY OF THE INVENTION 
The present invention has been made to solve the 
problems of prior arts described above, and its object 
5 is to provide an image processor and an image 

processing method for embedding a registration signal 
more appropriately . 

For this purpose, the image processor according to 
the present invention uses an irrotationally symmetric 
10 pattern arrangement to embed in image data the digital 
watermark including position and rotation information. 

Also, another image processor of the present 
invention is an image processor capable of extracting 
digital watermark information from image data in which 
15 the above described digital watermark information 

including position and rotation information is embedded, 
comprising : 

a rotation information searching means for 
performing processing for extracting the above 
20 described rotation information from the image data with 
the above described digital watermark information 
embedded therein, for a plurality of rotation angles 
different from one another; 

a position information searching means for 
25 performing processing for extracting the above 

described position information from the above described 
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image data, for a plurality of start-of-extraction 
positions different from one another; 

a calculating means for calculating confidence 
coefficients indicating accuracy as to whether the 
5 position and rotation information has been extracted, 
for each information searched by the above described 
rotation information searching means and position 
information searching means and extracted as position 
and rotation information; and 

10 a determining means for determining the position 

and rotation angle at which the above described digital 
watermark information is embedded in the above 
described image data, based on the confidence 
coefficients calculated by the above described 

15 calculating means. 

Furthermore, the present invention includes image 
processing methods, program products and computer data 
signals, capable of achieving the functions of the 
image processor. 

20 Other features and advantages of the present 

invention will be apparent from the following 
description taken in conjunction with the accompanying 
drawings, in which like reference characters designate 
the same or similar parts throughout the figures 

25 thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 
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FIG. 1 is a block diagram showing a general 
configuration of an digital watermark embedding 
apparatus of the preferred embodiment of the present 
invention; 

5 FIG. 2 is a block diagram showing a general 

configuration of an digital watermark extracting 
apparatus of the preferred embodiment of the present 
invention; 

FIG. 3 shows one example of image data generated 
10 at the extraction side in print-system processing; 

FIG. 4 is a block diagram showing a detailed 
configuration of registration a signal embedding unit 
of the preferred embodiment of the present invention; 
FIG. 5 is a view for illustrating registration 
15 signals of the preferred embodiment of the present 
invention; 

FIG. 6 shows a detailed configuration of a 
confidence distance computing unit; 

FIG. 7 is a block diagram showing a details of a 
20 scale matching unit of the preferred embodiment of the 
present invention; 

FIGS. 8A and 8B illustrate the extraction of the 
registration signal of the preferred embodiment of the 
present invention; 
25 FIG. 9 shows a pattern arrangement that is used 

when added information is embedded and is extracted by 
the preferred embodiment of the present invention; 
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FIG. 10 is a block diagram showing a details of an 
added information embedding unit of the preferred 
embodiment of the present invention; 

FIG. 11 is a block diagram showing a details of 
5 the embedment position determining unit in the 
preferred embodiment of the present invention; 

FIG. 12 shows occurrence frequency distribution of 
a blue noise mask (a cone mask) ; 

FIG. 13 is an illustration of spatial frequency 
10 properties of the visual sense of human beings; 

FIGS. 14A and 14B show spatial frequency 
properties of the blue noise mask and the cone mask; 

FIG. 15 is a simple illustrative view of a 
position reference mask of the preferred embodiment of 
15 the present invention; 

FIG. 16 is a conceptual view showing the embedment 
position in the position reference mask; 

FIGS. 17A and 17B show each pattern arrangement 
being developed on the mask shown in FIG. 16; 
20 FIGS. 18A and 18B show operations of embedding 

repeatedly the minimum embedment unit of added 
information Inf in the entire image of the preferred 
embodiment of the present invention; 

FIG. 19 illustrates computation for embedding 
25 added information Inf of the preferred embodiment of 
the present invention; 
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FIG. 20 illustrates an added information 
extracting unit of the preferred embodiment of the 
present invention; 

FIG. 21 illustrates added information Inf being 
5 extracted by the preferred embodiment of the present 
invention; 

FIG. 22 shows extraction being performed despite 
absence of added information Inf of the preferred 
embodiment of the present invention; 
10 FIG. 23 shows ideal occurrence frequency 

distribution when a confidence distance d is extracted 
from an original image; 

FIG. 24 shows the case where the confidence 
distance d is extracted from the image in which an 
15 digital watermark is embedded; 

FIG. 25 illustrates an example of the occurrence 
frequency distribution of the confidence distances dl, 
d2 of the preferred embodiment of the present 
invention; 

20 FIG. 26 illustrates the principle of the embedment 

and extraction of the registration signal of the 
preferred embodiment of the present invention; 

FIGS. 21 A to 27C show a concept for retrieval of a 
leading position of embedment of the preferred 

25 embodiment of the present invention; 
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FIG. 28 is a flowchart illustrating registration 
processing of the preferred embodiment of the present 
invention; 

FIG. 29 is a block diagram showing a configuration 
5 of another embodiment of a registration signal 
embedding unit in a spatial domain; 

FIG. 30 illustrates two sets in a patchwork method 
of the preferred embodiment of the present invention; 

FIG. 31 is a flowchart for illustrating a total 
10 control of a digital watermark embedding apparatus of 
the preferred embodiment of the present invention; 

FIG. 32 is a flowchart for illustrating a total 
control of an digital watermark extracting apparatus of 
the preferred embodiment of the present invention; 
15 FIGS. 33A and 33B show an example of a pattern 

arrangement orthogonal to the pattern shown in FIG. 9; 

FIGS. 34 illustrates an "orthogonal" pattern 
arrangement ; 

FIGS. 35A and 35B show first and second position 
20 reference masks of the preferred embodiment of the 
present invention; 

FIG. 36 shows a configuration of added information 
Inf of the preferred embodiment of the present 
invention; 

25 FIG. 37 shows one example of each coefficient of 

the blue noise mask of the preferred embodiment of the 
present invention; 
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FIG. 38 shows one example of each coefficient of 
the cone mask of the preferred embodiment of the 
present invention; 

FIG. 39 shows chromaticity spatial number 
5 properties of the visual sense of human beings; 

FIG. 40 shows a minimum coding unit in a JPEG 

mode ; 

FIG. 41 shows sampling of brightness and color- 
difference signals in the JPEG mode; 

10 FIG. 42 shows positive and negative operational 

areas (patches) of the pattern arrangement; 

FIG. 43 shows correspondences between the gray 
scale expressed by the area gray scale and the gray 
scale expressed by the gray-level gray scale of the 

15 preferred embodiment of the present invention; 

FIG. 44 shows a principle of gray scale 
information propagating before and after processing of 
conversion of gray scales of the preferred embodiment 
of the present invention; 

20 FIGS. 45A and 45B illustrate a difference in 

halftone processing of a printer depending on 
resolutions of images; 

FIG. 46 illustrates changes in ink dots depending 
on the embedment of the patch; 

25 FIG. 47 illustrates an increase in ink dots 

depending on the size and depth of the patch according 
to a preferred embodiment of the present invention; 
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FIGS. 48A and 48B show a difference in pattern 
arrangement units between the positive patch and the 
negative patch, depending on the difference in 
resolutions of images ; 
5 FIGS. 49A to 49D show pattern arrangements for 

embedding added information Inf corresponding to each 
resolution of the image in the preferred embodiment of 
the present invention; 

FIGS. 50A and SOB show shifts of impulse signals 
10 of amplitude spectra by scaling in the preferred 
embodiment of the present invention; 

FIGS. 51A and 51B show shifts of impulse signals 
of amplitude spectra by rotation in the preferred 
embodiment of the present invention; 
15 FIG. 52 shows the pattern arrangement and an area 

of overlapping patches of the rotated pattern 
arrangement in the preferred embodiment of the present 
invention; and 

FIG. 53 shows relationship between a rotation 
20 angle \|/ and the area of the shaded portion in FIG. 52 
in the preferred embodiment of the present invention . 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Preferred embodiments of this invention will be 
25 illustratively described in detail below, referring to 
the drawings. However, the relative placement of 
components , values, formulae and the like described in 
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these embodiments are not intended to limit thereto the 
scope of this invention. 
< First embodiment > 

[1. Overview of digital watermark embedding 
5 apparatus] 

First, the outline of an digital watermark 
embedding apparatus will be described as a preferred 
embodiment of the present invention, referring to FIG. 
1. FIG. 1 is a block diagram showing a general 
10 configuration of the digital watermark embedding 
apparatus . 

As shown in FIG. 1, the digital watermark 
embedding apparatus includes a color component 
extracting unit 0101, a registration signal embedding 

15 unit 0102, a pattern arrangement determining unit 0110, 
an embedment position determining unit 0103, an added 
information embedding unit 0104, a color component 
synthesizing unit 0105, a JPEG compressing and encoding 
unit 0106, a memory 0107, a JPEG expanding and decoding 

20 unit 0108 and a printer 0109. 

Image data I is inputted in this digital watermark 
embedding apparatus. The image data I that is inputted 
is multi-valued image data to which a predetermined 
plurality of bits is assigned to one pixel. 

25 Furthermore, in this embodiment, image data I that is 
inputted, whether it is gray scale image data or color 
image data, can be adapted. 
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The gray scale image data is configured such that 
one type of components constitute one pixel, and the 
color image data is configured such that three types of 
components constitute one pixel. In this embodiment, 
5 these three types of components are a red color 

component (R) , a green color component (G) and a blue 
color component (B) . However, the present invention is 
applicable to a combination of other color components. 

The image data I inputted in the digital watermark 

10 embedding apparatus is first inputted in the color 

component extracting unit 0101. In the case where the 
inputted image data I is color image data, only the 
blue color component is separated from this image data 
by the color component extracting unit 0101, and is 

15 outputted to the registration signal embedding unit 
0102 in the subsequent stage. 

On the other hand, other color components are 
outputted to the color component synthesizing unit 0105 
in the subsequent stage. That is, at this time, only 

20 the color component to which digital watermark 

information is to be embedded is separated and sent to 
an digital watermark processing system. 

In this embodiment, embedment of digital watermark 
information is performed for the blue color component. 

25 This is due to the fact that of the red color, blue 
color and green color components, the blue color 
component is the most insensitive to the visual sense 
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of human beings. Thus, the embedding of digital 
watermark information in the blue color component is 
effective in making degradation of image quality due to 
digital watermark information less perceptible by human 
5 eyes, compared to the embedding of digital watermark 
information of other components. 

Also, in the case where the inputted image data I 
is gray scale image data, the color component 
extracting unit 0101 transforms the gray scale image 

10 data into pseudo color image data on a temporary basis. 

The pseudo color cited herein is color image data 
configured such that three types components constitute 
one pixel, but in this embodiment, it is image data 
with the values of three types of components (R, G, B) 

15 being equal to one another. 

The gray scale image data is transformed into the 
aforesaid pseudo color image data, and the blue color 
component (B) in this color image data is extracted and 
outputted to the registration signal embedding unit 

20 0102. 

On the other hand, other color components are 
outputted to the color component synthesizing unit 0105. 
In this way, as in the case of the color image data 
described above, digital watermark information is 
25 embedded in only the blue color component, not in all 
the color components. 
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Furthermore, in the following description, 
explanation will be presented such that cases where the 
image data I is color image data and cases where the 
image data I is gray scale data are not distinguished 
5 where possible. That is, explanation will be presented 
such that color image data and pseudo color image data 
are not distinguished. 

The registration signal embedding unit 0102 will 
be described. The registration signal is a signal 
10 required for carrying out geometric correction as 
preprocessing for extraction of digital watermark 
information. 

Image data of the blue color component obtained 
with the color component extracting unit 0101 is 

15 inputted in the registration signal embedding unit 0102. 
The registration signal embedding unit 0102 embeds the 
registration signal in image data using a type of 
digital watermark technique. That is, in the image 
data in which the registration signal is embedded, the 

20 registration signal is not perceptible by the visual 
sense of human beings. The registration signal 
embedding unit 0102 outputs the image data in which the 
registration signal is embedded. 

The pattern arrangement determining unit 0110 

25 determines a pattern arrangement for embedding digital 
watermark information (added information) based on the 
resolution of the image expressed by image data to be 
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inputted and the output resolution from the printer so 
that digital watermark information (added information) 
is adequately extracted (detected) even if the gray- 
level gray scale is changed to the area gray scale when 
5 the image data in which digital watermark information 
is embedded is printed with the printer 0109. 

Furthermore, the resolution of the image is 
defined as the number of pixels per inch of the image 
(bit map image) in the case of printing this image in 

10 predetermined size. Thus, in the case of printing an 
image in a predetermined size, as the number of pixels 
of the image is increased, the resolution of the image 
is enhanced. As a unit for expressing the resolution 
of images, pixel/inch is used. 

15 Also, the output resolution of the printer 

represents the number of dots per inch printed on a 
print medium by the printer. As the number of dots per 
inch printed by the printer is increased, the 
resolution of the printer is enhanced. 

20 The pattern arrangement determining unit 0110 

outputs to the embedment position determining unit 0103 
the pattern arrangement selected from a plurality 
thereof, along with the inputted image data. The 
embedment position determining unit 0103 determines an 

25 embedment position of added information Inf in the 

image data in which the registration signal is embedded. 
The embedment position determining unit 0103 also 
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outputs to the added data information embedding unit 
0104 control data expressing a position at which added 
information Inf is embedded in the image, along with 
the inputted data and the pattern arrangement. 
5 The added information embedding unit 0104 inputs 

added information Inf (a plurality. of bit information) 
in addition to the aforesaid image data, pattern 
arrangement and control data. This added information 
Inf is embedded at the aforesaid determined embedment 

10 position in the image data of the blue color using an 
digital watermark technique. The image data in which 
added information Inf is outputted by the added 
information embedding unit 0104, and is inputted in the 
color component synthesizing unit 0105. 

15 The color component synthesizing unit 0105 uses 

the blue color component processed by the previous 
stages (up to added information embedding unit 0104) 
and the red color and green color components inputted 
directly from the color component extracting unit 0101 

20 to synthesize a form of normal color image data. 

The color image data obtained by the color 
component synthesizing unit 0105 is then outputted to 
the JPEG compressing and encoding unit 0106. The JPEG 
compressing and encoding unit 0106 transforms color 

25 image data comprised of the inputted red color, blue 

color and green color components into color image data 
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comprised of the brightness and color different 
components to perform JPEG compression and coding. 

JPEG compressed data compressed by the JPEG 
compressing and encoding unit 0106 is stored in the 
5 memory 0107 on a temporary basis. Then the JPEG 

compressed data is read from this memory at the timing 
when it is sent to an external apparatus or it is 
printed, and is outputted to the JPEG expanding and 
decoding unit 0108. The JPEG expanding and decoding 

10 unit 0108 expands the aforesaid JPEG compressed data 
and outputs it as color image data. 

The color image data wl outputted from the JPEG 
expanding and decoding unit 0108 is inputted in the 
printer 0109. The printer 0109 transforms the inputted 

15 color image data into color components of CMYK, and 
subjects it to halftone processing, followed by 
outputting it as a printed matter pwl on a print medium 
such as paper. 

Furthermore, there is a possibility that this 

20 printed matter pwl is passed to a person other than the 
user of this apparatus and is subjected to attacks such 
as geometric manipulation including rotation or copying 
with a copying machine. Assume that the aforesaid 
printed matter that could possibly be distorted is a 

25 printed matter pwl 1 . This printed matter pwl 1 will be 
digitized again using a scanner 0201 shown in FIG. 2. 
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A general control of the digital watermark 
embedding apparatus of this embodiment comprising each 
unit described above will be described below, referring 
to^a flowchart shown in FIG. 31. 
5 First, in Step 3102, image data I is inputted in 

the color component extracting unit 0101. This also 
includes a process of reading photographs and printed 
matters with a scanner or the like to generate image 
data. Furthermore, the color component extracting unit 

10 0101 separates the blue color component from the 

inputted image data I to use the same as input of the 
registration signal of the subsequent stage. 

Then, the registration signal is generated in Step 
3103, and this registration signal is embedded in Step 

15 3104. This registration signal embedment processing in 
Step 3104 corresponds to the processing that is 
performed within the registration signal embedding unit 
0102 shown in FIG. 1. 

In Step 3111, a pattern arrangement is determined 

20 by the pattern arrangement determining unit 0110. The 
pattern arrangement is used for embedding added 
information, in accordance with the output resolution 
of the printer 0109 and the resolution of the image. 

Furthermore, a mask is created in Step 3105, and 

25 the mask created in Step 3105 is inputted in a mask 
referring unit in Step 3106 to define relationship 
between embedment bit information and the embedment 
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position. In step 3107, the pattern arrangement 
determined in Step 3111 is inputted, and the mask is 
extended to an extended mask referring also to this 
pattern arrangement . 
5 In Step 3108, added information Inf is embedded in 

the image data in which the registration signal is 
embedded in steps 3103 and 3104. In this added 
information embedment processing, added information Inf 
is repeatedly embedded in the entire image in units of 

10 macro blocks. This processing will be described in 

detail with FIG. 10 described later. Here, the macro 
block refers to a minimum embedded unit, and all the 
information of one integral added information Inf is 
embedded in an image area corresponding to this macro 

15 block. 

In step 3109, the data in which added information 
inf is embedded is subjected to JPEG compression and 
coding and stored in the memory 0107, and is further 
subjected to JPEG expansion and decoding, and is then 
20 outputted as the printed matter pwl from the printer 
0109. 

[2. Digital watermark extracting apparatus] 
Now, the outline of the digital watermark 
extracting apparatus in this embodiment will be 
25 described referring to FIG. 2. FIG. 2 is a block 

diagram showing a general configuration of the digital 
watermark extracting apparatus in this embodiment. 
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As shown in FIG. 2, the digital watermark 
extracting apparatus of this embodiment is comprised of 
the scanner 0201, a color component extracting unit 
0202, a registration unit 0203 and an added information 
5 extracting unit 0204. First, the printed matter pwl 1 
is set on a manuscript stage of the digital watermark 
extracting apparatus, and the scanner 0201 scans this 
printed matter pwl 1 , whereby digitized image data wl' 
is generated. As described above, there is a 

10 possibility that the pwl' is different from the printed 
matter pwl in FIG. 1. 

The image data wl 1 has experienced attacks causing 
a variety of geometric distortions to the image data wl 
Attacks include scaling, rotation, printing & scanning 

15 and the like, but in the case of this embodiment, 

attacks involving at least one-time printing & scanning 
Therefore, ideally, the image data wl 1 and wl are 
identical with each other in contents, but in fact 
these two image data are often significantly different 

20 in contents from each other. 

Thus, the color component extracting unit 0202 
inputs image data wl 1 and extracts the blue color 
component, followed by outputting the image data of the 
blue color component to the registration unit 0203 in 

25 the subsequent stage. The red color and green color 

components other than the blue color in the image data 
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wl ' are not required, and are thus discarded at this 
time . 

Image data wli' of the blue color component 
obtained by the color component extracting unit 0202 is 
5 inputted by the registration unit 0203. Then this 

image data wli 1 of the blue color is used to generate 
image data wl 2 ' with geometric distortions corrected. 

While the image data wl 1 may have a scale and 
rotation different from those of the image data wl as 

10 described above, the image data wl 2 ' is always 

identical in scale to the image data wl . A reason for 
this and detailed processing for making the image data 
WI2 f identical to the image data wl in scale will be 
described later. 

15 The added information extracting unit 0204 is 

subjected to predetermined processing adapted for the 
embedment mode in the added information embedding unit 
0103, whereby added information Inf embedded in the 
image data wl 2 ' can be extracted and the embedded added 

20 information Inf is outputted. 

The general control of the digital watermark 
extracting apparatus of this embodiment comprising each 
configuration described above will be described 
referring to the flowchart shown in FIG. 32. 

25 First, in Step 3202, image data wl' is inputted. 

This image data wl ' is obtained by scanning image data 
that is presumably the printed matter pwl with the 
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scanner. Generally, the image data wl 1 is 
significantly different from the image data wl. Also, 
only the blue color component of this image data wl 1 is 
extracted and is used in the next step. 
5 In next Step 3203, the scale of the inputted image 

data wli 1 of the blue color component is corrected, 
whereby it is corrected to a rotation angle with 
possibility of extraction. This affine transformation 
correction processing is a processing that is carried 

10 out in the registration unit 0203 in FIG. 2, and 

detailed description thereof will be presented later. 

In next step 3211, a scaling rate outputted from 
step 3203 is used to determine the pattern arrangement 
used for embedding added information Inf. Then, in 

15 Step 3204, the rotation angle of the inputted image 

data wli' of the blue color component is corrected, and 
the offset thereof is corrected too. 

Extraction processing using a first pattern 
arrangement is performed in Step 3206 and extraction 

20 processing using a second pattern arrangement is 

performed in Step 3205, and thus form the image data 
wl 2 * with scale, rotation and offset already corrected 
is extracted the added information Inf embedded therein, 
respectively . 

25 In Statistical testing step 3207, correctness of 

the aforesaid extracted added information Inf is 
calculated and determined. If it is determined in this 
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Statistical testing step 3207 that the added 
information Inf is not correct, a return to Step 3202 
is made, and an image in which the added information 
Inf is presumably embedded is inputted again. 
5 On the other hand, if it is determined in the 

Statistical testing step 3207 that the added 
information Inf is correct, advancement to Step 3208 is 
made, and the added information Inf is extracted. Also, 
f5 » in Step 3210, information indicating the aforesaid 

15 10 correctness is displayed as a reliability index D 

§ ~ described later. 

2 Pattern arrangement determination processing, 

tn offset alignment processing, extraction processing 

C3 using the first pattern arrangement, extraction 

□ 15 processing using the second pattern arrangement, 

f3 statistical testing processing and comparison 

processing as described above are processes that are 
performed within the added information extracting unit 
0203 in FIG. 2, and detailed description thereof will 
20 be presented later. 

[3. Detailed description of each part] 
Each part of this embodiment of which outline has 
been described will be now described in detail. First, 
the registration unit 0203 at the extracted side of the 
25 digital watermark, and processing called registration 
that is performed in Step 3203 will be described. 
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The registration processing is preprocessing 
allowing added information Inf to be extracted from 
image data wl 1 inputted in the digital watermark 
extracting apparatus at the time of extracting digital 
5 watermark information. Generally, the term 

"registration processing" not only means the affine 
transformation processing but also implicates position 
alignment processing and the like. 

In this embodiment, however, the position 
10 alignment processing uses position information embedded 
as part of added information Inf, and is preformed in 
the added information extracting unit 0204. 

In the following, first, how the image data 
subjected to processing of the print-system changes 
15 will be considered. Then, registration processing for 
such changes will be examined, and registration 
processing on the print-system will be considered. 

This embodiment in which the image data wl is 
printed by a YMCK ink jet printer and this printed 
20 matter is scanned with a scanner is discussed. 

At this time, in the case where the output 
resolution from the printer and the input resolution 
from the scanner are different from each other, the 
scale of the original image data wl is different from 
25 that of the image data wl' obtained through scanning. 
Thus, the possibility that digital watermark 
information can be correctly extracted directly from 
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the obtained image data wl' is low. Therefore, it is 
necessary to provide a unit capable of correcting a 
difference between these scales. 

In this embodiment, since both the input 
5 resolution and output resolution are known, a scale 
ratio can be calculated from the ratio between these 
resolutions. For example, if the output resolution is 
600 dpi and the input resolution is 300 dpi, the scale 
ratio of the image before printing to the image after 

10 scanning is 1:2. Thus, scaling is applied to the image 
data wl 1 using an appropriate scaling algorithm, in 
accordance with the calculated scale ratio. This 
enables the image sizes expressed by the image data wl 
and the image data wl 1 to be adjusted to the same scale. 

15 However, input and output resolutions are not 

necessarily known in all cases. In a case where both 
the resolutions are not known, the above described 
method cannot be used. In this case, in addition to a 
unit for correcting difference in scales, a unit for 

20 determining the scale ratio is further required. 

Also, an image with the image data wl subjected to 
processing of the print-system will be an image as 
shown in FIG. 3 after it is inputted through scanning 
of the scanner. In FIG. 3, the whole of 0301 is an 

25 image expressed by the image data wl ' . This image data 
0301 is comprised of an original image 0302 expressed 
by the image data wl and a white margin 0303. A margin 
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like this, if cut by a user using a mouse and the like, 
results in incorrectness. Position alignment 
processing for position misalignment caused by scanning 
is performed through offset alignment processing in 
5 added information extraction processing 0204. 

Also, if meticulous attention is not paid, 
rotation is applied to the original image 0302 shown in 
FIG. 3. Thus, in order that the digital watermark can 
be extracted irrespective of placement, it is necessary 

10 to provide a unit for knowing a rotation angle in 

addition to a unit for correcting the rotation applied 
to the image, as in the case of scaling. Determination 
and correction of the rotation angle are performed by 
registration processing and offset matching processing. 

15 It can be considered that a problem like this is 

inevitably occurs for the image data wl 1 obtained 
through the print-system or the read-system. In the 
case where the image data wl is subjected to processing 
of the print-system, these problems should be solved. 

20 Up to this point, cases where image data is 

obtained through at least onetime processing of the 
print-system before the digital watermark is extracted 
have been described, but a state like this can also 
occur from man-made edition. 

25 [3-1. Registration signal embedment processing] 

The registration signal embedding unit and the 
registration unit in this embodiment provided for 
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solving the above described problems which arise due to 
the difference in scale and the rotation will be 
described below, assuming that the ratio between input 
and output resolutions is unknown. 
5 First, the registration signal embedding unit 0102 

(Step 3104) will be described in detail. 

The registration signal embedding unit 0102 is 
situated more to the front than the added information 
embedding unit 0104. This unit 0102 embeds in the 

10 original image data, in advance the registration signal 
referred to the registration of the image data wl 1 in 
the registration unit in FIG. 2. This registration 
signal is embedded in image data (the blue color 
component of color image data in the case of this 

15 embodiment) so that it is less perceptible by human 
eyes as digital watermark information. 

(First typical example of the embedding unit) 
An internal configuration of the registration 
signal embedding unit 0102 is shown in FIG. 4. The 

20 registration signal embedding unit 0102 includes a 

block dividing unit 0401, a Fourier transforming unit 
0402, an adding unit 0403, an inverse Fourier 
transforming unit 0404 and a block synthesizing unit 
0405. Details of each unit will be described below. 

25 The block dividing unit 0401 divides inputted 

image data into a plurality of blocks that does not 
overlap one another. The size of this block is defined 
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as a power of 2 in this embodiment. In fact, sizes 
other than this are applicable, but in the case where 
the size of the block is a power of 2, high-speed 
processing can be performed in the Fourier transforming 
5 unit 0402 that is coupled to the rear of the block 
dividing unit 0401. 

Blocks divided by the block dividing unit 0401 are 
grouped into sets Ii and I2, and of these, Ii is 
inputted in the Fourier transforming unit 0402 in the 

10 subsequent stage, and I2 is inputted in the block 
synthesizing unit 0405 in the subsequent stage. 

In this embodiment, one block situated nearest the 
center in the image data I is selected as Ii out of the 
blocks obtained by the block dividing unit 0401, and 

15 all the remaining blocks are selected as I2- 

This is due to the fact that this embodiment can 
be achieved by using at least one block, and the less 
the number of blocks, the more process time can be 
reduced. However, the present invention should not be 

20 limited to this, but as a matter of course, cases where 
two or more blocks are selected as Ii are also included 
in the category. 

Also, information as to sizes in which image data 
is divided into blocks, and which blocks are selected 

25 for embedment of the registration signal is shared by 
the digital watermark embedding apparatus and the 
digital watermark extracting apparatus. 
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Part of image data Ii obtained through dividing by 
the block dividing unit 0401 is inputted in the Fourier 
transforming unit 0402. 

The Fourier transforming unit 0402 transforms the 
5 inputted image data Ii by a Fourier transformation. 

While the original data form of the inputted image data 
ii is called a spatial domain, the data form after it 
is subjected to Fourier transformation is called a 
frequency domain. Fourier transformation is applied 
10 all the inputted blocks. 

Furthermore, in this embodiment, since the size of 
the block to be inputted is a power of 2, fast Fourier 
transformation is used for enhancing the speed of 
processing . 

15 While Fourier transformation requires a 

computation amount of nxn times, the fast Fourier 
transformation is a transformation algorithm that can 
be executed by a computation amount of (n/2) log 2 (n) 
times. Here, n is a positive integer. Fast Fourier 

20 transformation and Fourier transformation are different 
from each other in only the speed for obtaining results 
of computation, and the same result is obtained from 
the both. Therefore, in this embodiment, fast Fourier 
transformation and Fourier transformation are not 

25 distinguished in the present explanation. 

Furthermore, image data in the frequency domain 
obtained by Fourier transformation is represented by 
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amplitude spectra and phase spectra. Of these, only 
the amplitude spectra are inputted in the adding unit 
0403. On the other hand, the phase spectra are 
inputted in the inverse Fourier transforming unit 0404. 
5 The adding unit 0403 will be described. In the 

adding unit 0403, a signal r called a registration 
signal is separately inputted along with the aforesaid 
amplitude spectra. Examples of registration signals 
include an impulse signal 0502, 0503, 0504, and 0505 as 

10 shown in FIG. 5. 

In FIG. 5, the amplitude spectra of two 
dimensional spatial frequency components obtained 
through Fourier transformation are shown. The central 
one is a low frequency component and peripheral ones 

15 are high frequency components. 0501 is the amplitude 
spectrum of the signal component possessed by the 
original image component, and for signals corresponding 
to natural images such as photos, a large number of 
large signals are concentrated in the low frequency 

20 domain . On the other hand, few signals exist in the 
high frequency domain. 

Furthermore, in this embodiment, explanation is 
presented assuming that natural images are subjected to 
a series of processing, but the present invention 

25 should not be limited thereto, and document images, CG 
images and the like may be processed in a similar way. 
However, the form of this embodiment is especially 
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effective in the case of processing natural images 
having a relatively large quantity of medium 
concentration . 

FIG. 5 shows one example of this embodiment in 
5 which impulse signals 0502, 0503, 0504 and 0505 are 
added to lateral and vertical Nyquist frequency 
components of signals in the frequency domain for the 
signal 0501 that the natural image originally possesses. 
As in the case of this example, the registration signal 

10 is preferably a impulsive signal. This is because only 
the registration signal is easily extracted in the 
digital watermark extracting apparatus described later. 

In the example shown in FIG . 5, impulse signals 
are added to Nyquist frequency components of input 

15 signals, but the present invention should not be 

limited thereto. That is, those signals where the 
registration signal is not removed even if an image in 
which the digital watermark information is embedded is 
attacked, may be accepted. As described above, an 

20 irreversible compression system such as JPEG 

compression has a low-pass filter like effect. 
Therefore, there is a possibility that even though the 
impulse signal is embedded in the high frequency 
component being a subject of information compression 

25 here, the signal is removed through 
compression/expansion processing . 
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On the other hand, embedment of the impulse in the 
low frequency component has a disadvantage in that the 
signal is more perceptible as noise due to visual 
characteristics of human beings, compared to embedment 
5 in the high frequency domain. Thus, in this embodiment, 
the impulse signal is embedded in a medium frequency, a 
first frequency or higher where the signal is hardly 
perceptible by the visual sense of human beings and a 
second frequency or lower where the signal is not 

10 easily removed through irreversible 
compression /expansion processing . 

Also, this registration signal is added to each 
block (one block in the case of this embodiment) 
inputted in the adding unit 0403. The adding unit 0403 

15 outputs to the inverse Fourier transforming unit 0404 a 
signal with the registration signal added to the 
amplitude spectrum of image data in the frequency 
domain . 

The inverse Fourier transforming unit 0404 
20 subjects the inputted image data in the frequency 

domain to inverse Fourier transformation. This Fourier 
transformation is applied to all the inputted blocks. 
As in the case of the above described Fourier 
transforming unit 0402, since the size of the inputted 
25 block is a power of 2, fast Fourier transformation is 

used for enhancing the speed of processing. The signal 
in the frequency inputted in the inverse Fourier 
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transforming unit 0404 is transformed into a signal in 
the spatial domain through the inverse Fourier 
transformation, and is outputted. 

The image data in the spatial domain, which is 
5 outputted by the inverse Fourier transforming unit 0404 
is inputted in a block coupling unit 0405. The block 
coupling unit 0405 performs processing opposite to 
dividing performed by the block dividing unit 0405. As 
q a consequence of processing by the block coupling unit 

?fi 10 0405, image data (blue color component) is 

reconstructed and outputted. 
2 Up to this point, details of the first typical 

~ hr * example of the registration signal embedding unit 0102 

y have been described. 

S3 15 (Second typical example of the embedding unit) 

Q A system in which the registration signal is 

embedded in the Fourier transformation area has been 
described with FIG. 4. But the present invention 
should not be limited to this example, and on the other 
20 hand, a system in which the registration signal is 

embedded in the spatial domain can also be considered. 
As a second typical example, an alternative example in 
which embedment of signals is performed in the spatial 
domain like this will be described using FIG. 29. 
25 As shown in FIG. 29, a block dividing unit 2901, 

an adding unit 2902, block synthesizing unit 2903 and 
an inverse Fourier transforming unit 2904 are used, in 



- 33 - 



order to achieve a system in which the registration 
signal is embedded in the spatial domain. 

The block dividing unit 2901 and the block 
synthesizing unit 2903 operate as in the case of the 
5 block dividing unit 0401 and the block synthesizing 
unit 0405 in FIG. 4. Image data that is inputted in 
the registration signal embedding unit 0102 is first 
inputted in the block dividing unit 2901, and is split. 
Blocks obtained here are inputted in the adding unit 

10 2902. On the other hand, the registration signal r is 
inputted in the inverse Fourier transforming unit 2904, 
and is transformed into a signal r T by through inverse 
Fourier transformation processing. At this time, the 
registration signal r' is a signal on the frequency 

15 domain as in the case of those shown in FIG. 5. 

Blocks from the block dividing unit 2901 and the 
signal r 1 from the inverse Fourier transforming unit 
2904 are inputted in the adding unit 2902, and are 
added respectively. The signal outputted from the 

20 adding unit 2902 is inputted in the block synthesizing 
unit 2 903, and image data (blue color component) is 
reconstructed and outputted. 

The configuration shown in FIG. 29 is to perform 
processing as in the case of FIG. 4 in the spatial 

25 domain, but is capable of performing more speedy 

processing because it requires no Fourier transforming 
unit, compared to the configuration in FIG. 4. 



- 34 - 



Furthermore, in FIG. 29, the signal r' is a signal 
independent of input image data I. Therefore, 
calculation of r', namely processing by inverse Fourier 
transforming unit 2904 does not need to be carried out 
5 each time the image data I is inputted, and r 1 can be 
generated in advance. Registration processing of 
referencing this registration signal will be described 
later . 

Q <<Patchwork method>> 

m 10 In this embodiment, a principle called a patchwork 

"p method is used for embedment of added information Inf. 

_f» Thus, first, the principle of the patchwork method for 

l''' use in this embodiment will be described. 

b J In the patchwork method, images are biased 

^ 15 statistically, thereby achieving embedment of added 

=3 information Inf. This will be described using FIG. 30. 

In FIG. 30, 3001 and 3002 refer to subsets, 
respectively, and 3003 refers to the whole image. Two 
subsets, A3001 and B3002 are selected from the whole 
20 image 3003. 

Selection of these two subsets allows embedment of 
added information Inf to be achieved through the 
patchwork method in this embodiment, if they do not 
overlap one another. However, the size and selection 
25 of these subsets have a significant influence on the 
resistance of added information Inf embedded through 
this patchwork method, namely strength such that the 
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added information is not lost when image data wl is 
attacked. This will be described later. 

Now, assume that the subsets A and B are sets 
constituted by N elements expressed by A= {ai, a2, . 
a N } and B={bi, b2, b N }, respectively. Each element, 

ai, bi of the subset A and the subset B is a image 
value or a set of image values. In this embodiment, 
they correspond to part of the blue color component in 
color image data. 

At this time, the following index d is defined. 

d=(l/N)21(ai-bi) 

This shows an expected value of a difference 
between image values of two sets. 

Selecting appropriate subsets A and B and defining 
an index d for a general natural image results in d~0 . 
Hereinafter, d is called a confidence distance. 

On the other hand, as an operation of embedding 
each bit constituting added information Inf, an 
operation of 

a r i=ai+c 

b'i^i-c 

is performed. This is an operation of adding c to all 
the elements of the subset A and subtracting c from all 
the elements of the subset B. 

At this time, in a manner similar to the previous 
case, subsets A and B are selected from an image in 




which added information Inf is embedded, and an index d 
is calculated. Then the result is 
d =(l/N)Z(a , i-b , i ) 

= (1/N)Z{ (a'i+c) - (b'i-c) } 
5 =(l/N)Z(a , i -b , i)+2c 
=2c, 

which is not 0. 

That is, when a certain image is given, the 
confidence distance d is calculated for the image, 

10 whereby it can be determined that added information Inf 
is not embedded if d~0, while it can be determined that 
added information Inf is embedded if d is a value 
different from 0 by a certain amount. Basic concept 
has been described up to this point. 

15 In this embodiment, information of a plurality of 

bits is embedded, applying the principle of this 
patchwork method. In this method, a method of 
selecting subsets A and B is defined by the pattern 
arrangement . 

20 In the aforesaid method, elements of the pattern 

arrangement are added to and subtracted from 
predetermined elements of the original image, thereby 
achieving embedment of added information Inf. 

An example of a simple pattern arrangement is 

25 shown in FIG. 9. FIG. 9 shows a pattern arrangement 

showing the amount of changes in pixel values from the 
original image in the case of referring to 8x8 pixels 
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for embedding one bit. As in FIG. 9, the pattern 
arrangement is comprised of arrangement elements having 
positive values, arrangement elements having negative 
values and arrangement elements having values of 0. 
5 In the pattern of FIG. 9, positions shown by 

arrangement elements of +c represent positions for 
increasing by c the pixel value at the corresponding 
position, and are positions corresponding to the 
m aforesaid subset A. On the other hand, positions shown 

y£ 10 by arrangement elements of -c represent positions for 

decreasing by c the pixel value at the corresponding 
position, and are positions corresponding to the 
EH aforesaid subset B. Also, positions shown by 0 

q represent positions other than those of the subsets A 

q 15 and B. 

In this embodiment, the number of arrangement 
elements having positive values and the number of 
arrangement elements having negative values is the same 
in order that the general concentration of the image is 
20 not changed. That is, the sum' of all arrangement 
elements results in 0 in one pattern arrangement. 
Furthermore, this condition is essential at the time of 
operation of extracting added information Inf as 
described later. 
25 Operation of embedding each bit constituting added 

information is performed using a pattern arrangement as 
described above. 
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In this embodiment, the pattern of FIG. 9 is 
placed several times in areas different from one 
another in the original image to increase/decrease the 
pixel value, thereby embedding bid information, namely 
5 added information Inf. In other words, not only a 
combination of a subset A and a set B, but also a 
plurality of combinations like a subset A 1 and a set B 1 , 
a subset A" and a set B" ... is assumed in areas of one 
r=l image that are different form one another, thereby 

fi 10 embedding added information constituted by a plurality 

£ of bits. 

"2 Furthermore, in this embodiment, added information 

- ?n Inf is repeatedly embedded in the case where the 

original image is large. This is due to the fact that 
S3 15 quantity large enough for statistical nature to appear 

£□ is required because the patchwork method uses 

statistical nature . 

Also, in this embodiment, in order that areas for 
which the pixel value is changed using the pattern 
20 arrangement do not overlap one another when a plurality 
of bits are embedded, relative positions at which the 
pattern arrangement is used are predetermined among 
mutual bits. That is, relationship between the 
position of the pattern arrangement for embedding first 
25 bit information constituting added information Inf and 
the position of the pattern arrangement for embedding 
second bit information is appropriately defined. 
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For example, if added information Inf is 
constituted by 16 bits, the positional relationship 
among respective 8x8 pixel pattern arrangements of 
first to sixteenth bits is provided in a relative 
5 manner so that degradation of image quality is reduced 
on an area having a size larger than 32x32 pixels. 

Furthermore, in the case of large image data, the 
aforesaid added information Inf (each bit information 
constituting this) is embedded in repetition as many 
10 times as possible. Its purpose is to enable each bit 
of added information Inf to be extracted correctly. 
Particularly in this embodiment, since statistical 
measurement taking advantage of the same added 
information Inf being embedded in repetition is 
15 performed, the aforesaid repetition is important. 

Selection of the embedment position as described 
above is carried out by the position determining unit 
0103 in FIG. 1. 

Next, how to determine the above-described subsets 
20 A and B will be described. 

[3-2. Pattern arrangement determining unit] 

Now, how to determine the subset A and the subset 
B in this embodiment will be described. 

In the patchwork method, how to determine the 
25 subset A and the subset B strongly affects attack 

resistance of added information Inf and the quality of 
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the image in which the added information Inf is 
embedded. 

In this embodiment, image data wl in which added 
information Inf is embedded and which is subjected to 
5 JPEG compression and expansion is print-outputted by 
the printer in FIG. 1 and is inputted by the scanner 
0201 in FIG. 2 before becoming image data wl . Various 
attacks including print and scanning are applied to the 
image data wl during processes for obtaining the wl. 

10 What should be done in order that added 

information Inf embedded through the patchwork method 
in this embodiment has resistance to attacks will be 
discussed below. 

In the patchwork method, the shape of the pattern 

15 arrangement and the scale of values of elements 

represent parameters for determining a tradeoff between 
the embedment strength of added information Inf and the 
image quality of the image data wl . Therefore, as for 
whether added information Inf can be extracted after 

20 applying the aforesaid attacks, optimization can be 
achieved by manipulating these parameters. For this, 
more detailed description will be presented. 

In this embodiment, the fundamental positional 
relationship between an element ai of the subset A and 

25 an element bi of the subset B in the patchwork is fixed 
by a matrix for which one example is shown in FIG. 9. 
Furthermore, the elements a± and bi are not limited to 
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one pixel value, but may be a set of a plurality of 
pixel values, respectively. 

A plurality of the pattern arrangements are 
assigned in the image such that they do not overlap one 
5 another, and each assigned pixel in this image is 

changed in accordance with the value of the element of 
the pattern arrangement. 

If the subset of pixels for which the image is 
changed to positive values (+c) of the pattern 

10 arrangement is A, and the subset of pixels for which 
the image is changed to negative values (-c) of the 
pattern arrangement is B, then it can be understood 
that the patchwork method is applied. 

Furthermore, in the following description, a group 

15 of pixels having positive values (+c) of the pattern 
arrangement (corresponding to the position of the 
element a± of the subset) is called a positive patch, 
and a group of pixels having negative values (-c) 
(corresponding to the position of the element bi of the 

20 subset) is called a negative patch. 

Hereinafter, there may be cases where the positive 
and negative patches are indiscriminately used, but the 
patch in that case refers to the positive patch and 
negative patch or any one of them. 

25 If the size of each patch of the pattern 

arrangement for which one example is shown in FIG. 9 is 
increased, the resistance to added information Inf is 
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enhanced, and the image after the added information Inf 
is embedded therein is significantly degraded in terms 
of image quality compared to the original image, 
because the value of the confidence distance d in the 
5 patchwork method becomes larger. 

On the other hand, if the scale of the value of 
each element of the pattern arrangement is decreased, 
the resistance to added information Inf is reduced, and 
the image after the added information Inf is embedded 

10 therein is not so much degraded in terms of image 
quality, compared to the original image. 

In this way, optimization of the scale of the 
pattern arrangement shown in FIG. 9 and the scale of 
the value of elements (±c) of patches constituting this 

15 pattern is very important for the resistance of image 
data wl and the image quality. 

First, the size of the patch will be considered. 
The resistance to added information Inf embedded 
through the patchwork method is enhanced if the size of 

20 the patch is increased, while the resistance to added 
information Inf embedded through the patchwork method 
is reduced if the size of the patch is decreased. This 
is due to the fact that irreversible compression and 
the print-system have a low pass filter-like effect 

25 throughout processing. 

The signal biased for embedding added information 
Inf is embedded as a signal of the low frequency 
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component if the size of the patch is increased, while 
the signal biased for embedding added information Inf 
is embedded as a signal of the high frequency component 
if the size of the patch is decreased. 
5 There is a possibility that the added information 

Inf embedded as a signal of the high frequency 
component is subjected to processing of the print- 
system, thereby being subjected to low pass filter-like 
processing and erased. On the other hand, it is likely 

10 that the added information Inf embedded as a signal of 
the low frequency component is not erased and can be 
extracted even though it is subjected to processing of 
the print-system. 

From the above, in order that added information 

15 Inf has resistance to attacks, a larger size of the 

patch is more desirable. However, increasing the size 
of the patch means adding a signal of the low frequency 
component to the original image, which leads to 
increased degradation of image quality in image data wl . 

20 This is because visual properties of human beings have 
properties of VTF as shown by 1301 of FIG. 13. As is 
apparent from 1301 of FIG. 13, the visual property of 
human beings is relatively sensitive to noises of the 
low frequency component, but is relatively insensitive 

25 to noises of the high frequency component. Therefore, 
it is desired that the size of the patch be optimized 
for determining the strength of added information Inf 
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embedded through the patchwork method and the image 
quality in image data wl . 

Now, values of the patch (±c) will be considered. 
Assume that the value of each element (±c) 
5 constituting the patch is called "depth". The 

resistance of added information Inf embedded through 
the patchwork method is increased if the depth of the 
patch is increased, while the resistance of added 
information Inf embedded through the patchwork method 

10 is reduced if the depth of the patch is decreased. 

The depth of the patch is closely associated with 
the confidence distance d that is used when added 
information Inf is extracted. The confidence distance 
d is an operational value for extracting added 

15 information Inf, and this will be described in detail 

in the section of extraction processing, but in general, 
the confidence distance d is increased, making it 
easier to extract added information Inf if the depth of 
the patch is increased. On the other hand, if the 

20 depth of the patch is decreased, the confidence 

distance d is reduced, making it more difficult to 
extract added information Inf. 

From the above, the depth of the patch is also an 
important parameter for determining the strength of 

25 added information Inf and the quality of the image in 
which the added information Inf is embedded, and it is 
desirably optimized for use. The optimized size and 
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depth of the patch is always used, thereby making it 
possible to achieve embedment of added information Inf 
such that resistance to irreversible compression and 
attacks such as print is retained, and degradation of 
5 image quality is reduced. 

Now, the depth and size of a specific patch for 
use in this embodiment will be described. To simplify 
the following discussion, processing in the print- 
system is also simplified. Conversion of gray scales 

10 by halftone processing will be considered as an example 
of processing in the print-system. 

The halftone processing is to change a method of 
expressing gray scales as described above. The visual 
sense of human beings perceives the gray scale 

15 similarly before and after the halftone processing. 

However, an inputting unit such as a scanner does not 
have ambiguous perception like that of human beings, 
and does not necessarily perceive the gray scale 
similarly before and after the halftone processing. 

20 That is, the scanner does not recognize by itself 

whether the gray scale expressed by the area gray scale 
really has gray scale information expressed by the 
gray-level gray scale. 

Then, what halftone processing enables the area 

25 gray scale to express the gray scale expressed by the 
gray-level gray scale will be discussed. First, 
relationship in terms of gray scale expression between 



- 46 - 




the gray-level gray scale and the area gray scale by 
halftone processing will be considered. 

An example of relationship between a dither matrix 
of 4x4 and the gray scale that can be expressed by the 
5 matrix is shown in FIG. 43. FIG. 43 shows the gray 

scale in which the matrix is expressed by the area gray 
scale, and the gray scale expressed by the matrix is 
indicated by the number below. 

There exist 16 pixels in the 4x4 matrix. 4x4+1 = 

10 17 gray scales can be expressed by on/off of these 16 
pixels. Generally, mxn dots subjected to halftone 
processing can express (mxn+l) gray scales. 

This will be described using an example of FIG. 44. 
In FIG. 44, assume that a pixel 4403 is expressed by 

15 the gray-level gray scale having 0 to 16 dynamic ranges 
and its value is 8. Four pixels having the same value 
as that of this pixel are arranged vertically and 
laterally respectively to generate a block 4402 having 
a size of 4x4. 

20 The generated block is subjected to halftone 

processing using an appropriate dither matrix having a 
size of 4x4 to generate binary data 4403. This binary- 
valued data is sent to the printer and outputted. Then, 
it is inputted again at same resolution as output 

25 resolution of the printer through an image inputting 

unit such as the scanner. At this time, assuming that 
the ratio between the output resolution of dots of the 



- 47 - 



printer and the input resolution at which the scanner 
reads the pixel is 1 : 1, the pixel outputted by the 
printer and inputted by the scanner equals the binary 
data 4403. 

5 The image data generated in this way is binary 

data 4404. By a method using interpolation processing 
appropriate to the binary data 4404, the binary data is 
scaled down through scaling to 1/(4x4) of its original, 
and multivalued data 4405 is generated. The value of 

10 this multivalued data is 8. Furthermore, in the case 
where the scanner does not have such a high resolution 
that the binary data 4403 can be determined as the 
binary data 4404, the binary data 4403 is optically 
transformed into the multivalued data 4405. 

15 Up to this point, the state in which gray scale 

information expressed by the gray-level gray scale is 
transformed into the area gray scale, and after that, 
when it is expressed by the gray-level gray scale again, 
the gray scale information is correctly propagated has 

20 been described using FIG. 44. Generally, halftone 

processing is performed using the area gray scale in 
which one pixel expressed by mxn pixels, and 
interpolation processing where binary data of these mxn 
pixels is rendered as one pixel, whereby gray scale 

25 information is propagated. 

For this embodiment, in order to provide 
resistance to attacks including print and scanning, the 
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size and depth of the patch for use in embedment of 
added information Inf is designed considering the 
aforesaid relationship between the area gray scale and 
the gray-level gray scale. Furthermore, for this 
5 embodiment, cases where images of various sizes are 
outputted in a predetermined size by the printer are 
discussed. 

FIGS. 45A and 45B show a case where two images 
4501 and 4504 different in resolution from each other 

10 are outputted in dimensions 4503 and 4506 identical in 
size to each other by the printer. FIGS. 45A and B 
show a series of processing where the resolution of the 
image is low and where the resolution of the image is 
high, respectively . 

15 First, the images 4501 and 4505 are subjected to 

scale-up processing so that one pixel corresponds to 
one dot. For a scale-up method at this time, 
interpolation such as a nearest neighbor method is used. 
Furthermore, the nearest neighbor method is a method in 

20 which scale-up is performed by copying the same pixel 
values to neighboring pixels (In the case where the 
image has very high resolution, scale-down (thinning- 
out) can be also considered) . As a result thereof, the 
images 4501 and 4504 are scaled up to the images 4502 

25 and 4505, respectively. Thereafter, they are expressed 
by dots as printed matters such as 4503 and 4506 (image 
data for printing) through halftone processing. 
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For actual processing in the printer, CMYK 
conversion processing, color matching and the like are 
performed, but they are omitted here to simplify 
description . 

5 From FIGS. 45A and 45B, that can be understood 

that as the resolution of the image decreased, one 
image can be expressed by a increased number of dots, 
and as the resolution of the image is increased, one 
image must be expressed by a decreased number of dots. 

10 Now, it will be shown that influence of embedment 

by the patch is transmitted also in the case of 
conversion from the gray-level gray scale to the area 
gray scale. Furthermore, influence by the resolution 
of the image is not taken into consideration here for 

15 making description easily understood. 

4601 and 4605 of FIG. 46 are image areas (subset 
A) operating with the positive patch in the case of 
embedding added information Inf in an image, and 
represent states before halftone processing. 4603 and 

20 4606 of FIG. 46 are image areas (subset B) operating 

with the negative patch in the case of embedding added 
information Inf in an image, and represent states 
before halftone processing. 

4601 and 4603 of FIG. 46 show cases where 

25 embedment of added information Inf by the patch has not 
been performed, and 4 605 and 4 607 show cases where 
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embedment of added information Inf by the patch has 
been performed. 

Also, at this time, 4601, 4603, 4605 and 4607 all 
correspond to one pixel and one dot just before 
5 halftone processing is applied. 

Through halftone processing, images shown by 4601, 
4603, 4605 and 4607 of FIG. 46 are expressed by dots as 
area gray scales, as in the case of 4602, 4604, 4606 
and 4608. 

10 It can be said that in the case where embedment of 

added information Inf has not been performed, a 
difference in the number of ink dots between 4 602 and 
4604 is generally small. In the case where the image 
is large and an averaged difference in ink dots is 

15 calculated for each patch, it is almost 0. 

On the other hand, it can be considered that in 
the case where embedment of added information Inf has 
been performed, a difference in the number of ink dots 
between 4 606 and 4608 comes to appear. 

20 Also in the case where added information Inf is 

expressed by the area gray scale, increase and decrease 
in ink dots can be controlled through the design of the 
patch. It can be said that the patchwork method is 
capable of giving resistance to attacks from print and 

25 scanning. 

Also, from FIG. 46, it can be intuitively imagined 
that the number of ink dots increases if the area in 
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which the patch is embedded is increased, and the 
number of ink dots increases if the depth of the patch 
is increased. 

Relationship between the patch and increase in ink 
5 dots will be described using FIG. 47. FIG. 47 is a 

drawing showing variation in ink dots depending on the 
size and depth of the patch. In FIG. 47, the lateral 
axis represents coefficient values of the dither matrix 
for subjecting to halftone processing the subset A or 

10 the subset B with one pixel scaled up to one dot, and 
the vertical axis represents the occurrence frequency 
of coefficient values of the dither matrix. At the 
same time, to make description easily understood, the 
averaged pixel vales of the subset A or the subset B 

15 with one pixel scaled up to one dot, which is subjected 
to halftone processing are shown in the lateral axis. 

As shown in FIG. 47, it may be thought that 
generally coefficient values of the dither matrix have 
almost no biased values and the occurrence frequencies 

20 are almost the same for a large subset A or subset B. 
It can be understood that assuming that the 
averaged pixel value 4703 before embedment is changed 
to the averaged pixel value 4704 after embedment 
through the embedment of added information Inf, only 

25 the slanted-line portion 4702 has the number of ink 

dots increased with binary formation processing of the 
dither matrix. 
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That is, it can be understood that there is 
proportionality between the depth of the patch and an 
increase in ink dots. Also, in the case where the size 
of the patch is increased, the area of the sloped-line 
5 portion 4702 increases in the direction of occurrence 
frequency because the occurrence frequency of 
coefficient values of the dither matrix is further 
increased, and thus it can be understood that there is 

fl - also proportionality between the depth of the patch and 

JS 10 an increase in ink dots. 

'|j Considering the properties described above, for a 

^2 entire image, (1) the depth of embedment is 

Cn proportional to the number of dots on a printed matter. 

C3 (2) The size of the patch is proportional to the number 

fj 15 of dots on a printed matter. 

rj That is, assuming that a difference in the number 

of dots for the entire image between the area in which 
positive patches for the entire image are embedded and 
the area in which negative patches are embedded for the 
20 entire image, which is varied through embedment of the 
patch, is A(5, Ap is expressed by: 

Ap=2X0CXPAXC+y . . . (Equation 47-1) 

a is a proportionality constant, y is a margin, C 
is a depth of embedment, and PA is an area 
25 corresponding to one pixel and one image of the 

positive or negative patch for the entire image. Where, 
a, and y are numbers determined by experiments. 
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Furthermore, the principle of this (Equation 47-1) 
is applicable because the aforesaid (1) and (2) hold 
not only in halftone processing by the dither matrix 
but also in an error spreading method. 
5 In the (Equation 47-1) , the resolution of the 

image, the output resolution of the printer and the 
input resolution of the scanner are not considered. 
Cases where the resolution of the image, the output 
resolution of the printer and the input resolution of 
:^ 10 the scanner are different will be discussed below. 

!f In this embodiment, the input resolution of the 

: jj scanner is fixed at 600 ppi, which is enough resolution 

:D for a flat bed scanner, in order to retain a maximum 

;3 amount of information. 

pa 

;g 15 Now, the output resolution of the printer and the 

3 resolution of the image will be considered. 

As already described with FIG. 4-5, in the case 
where the image is printed, the number of dots 
expressing one pixel is determined by the resolution of 
20 the image. One example will be presented below. 

Assume that in FIG. 45A, the image 4501 is an 
image that has 1000x800 pixels, for example. If this 
image is printed with a printer having output 
resolution of 1200 dpi in both the main scanning 
25 direction and the sub scanning direction so that the 
length of the long side is not more than 5 inches, 
scale-up processing is performed just before halftone 
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processing so that one pixel is rendered as one dot, 
and the number of output dots of the long side is made 
to be 1200 dpixS = 6000 dot. 

Thus, the image 4501 is scaled up to the image 
5 4502 having 6000x4800 pixels. In the case where the 
gray scale of 4501 is reproduced from the image 4503 
subjected to halftone processing, one pixel is 
expressed by 6x6 dots. 

On the other hand, assume that the image 4504 in 

10 FIG. 45B is an image having 3000x2400 pixels. If this 
image is outputted in a same way with a printer having 
resolution of 1200 dpi such that the length of the 
longer edge is no more than 5 inch, it is scaled up to 
the image 4505 having 6000x4800 pixels in a similar way 

15 so that one pixel equals one dot, and then becomes the 
image 4506 subjected to halftone processing. One pixel 
of the image 4504 is expressed by 2x2 dots. 

Because it can be considered that the gray level 
of the ink dot is already fixed, the dynamic range of 

20 the gray-level gray scale that can be expressed with 
one pixel is large in the case where one pixel is 
expressed by 5x5 dots. On the other hand, in the case 
where one pixel is expressed by 2x2 dots, the dynamic 
range of the gray-level gray scale that can be 

25 expressed with one pixel is small. 

As will be described in detail in added 
information extraction processing, for the patchwork 
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method, (sum of pixel values of the area in which 
positive patches are embedded) - (sum of pixel values 
of the area in which negative patches are embedded) is 
calculated in each pattern arrangement unit when 
detection is performed, and an average value for each 
pattern' arrangement is determined for the whole image. 
This average value is called a confidence distance d, 
and the larger this confidence distance, the more 
reliably added information can be extracted. 

FIG. 48 graphically shows a difference between 
positive and negative patch areas in a pattern 
arrangement. FIG. 48A shows the case of low resolution, 
in which a reference numeral 4801 denotes a positive 
patch area and a reference numeral 4802 denotes a 
negative patch are, and FIG. 48B' shows the case of high 
resolution, in which a reference numeral 4803 denotes a 
positive patch area and a reference numeral 4804 
denotes a negative patch area. 

Furthermore, since the ink dot has a fixed gray 
level per dot, the confidence distance d has a dynamic 
range including large values in the case of 4801 and 
4802 because one pixel is constituted by a large number 
of ink dots. On the other hand, in the case of 4803 
and 4804, the confidence distance d does not have a 
dynamic range including large values because one pixel 
is expressed by a small number of ink dots. 



# 



Generally, in the case where one pixel is 
constituted by a small number of dots (the resolution 
of the image is high) , there may be cases where a large 
confidence distance d can not be obtained, thus making 
5 it impossible to extract added information Inf, because 
the dynamic range of the gray scale which one pixel has 
is small. 

Therefore, in the case where the resolution of the 
„ image is high, it is necessary to enlarge the area of 

10 the patch or increase the depth of embedment (±c) . 
**j Generally, it is preferable that the area of the patch 

"■^ is enlarged, because positional displacements rise a 

W problem during high-resolution output. 

£3 Assuming that the number of dots required for 

□ 15 detecting added information per pattern arrangement is 

q A|Jp and the number of pixels of the positive or 

negative patch is P, from (Equation 47-1) , the 
relationship between a depth of embedment of the patch 
C and the number of dots expressing one pixel, mxn is 
20 expressed by: 

App=2xaxPX(mXn)x(C/255) +y (Equation 47-2) 

Here, (mxn) x (C/255 ) shows that the number of dots 
increases by only mxn for one dot at the maximum even 
when the depth of embedment C is changed through up to 
25 maximum 255 gray scales. Also, a and y denote a 

proportionality constant and a margin respectively. 
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Furthermore, the number of dots mxn for 
reproducing one pixel can be determined by mxn = 
(output resolution of printer in main scanning 
direction/resolution of image) x (output resolution of 
5 printer in sub scanning direction/resolution of image) , 
and mxn decreases as the resolution of the image is 
increased. 

Thus, a and y are determined by experiments, and 
„ then the depth of embedment per pattern arrangement, 

jjf 10 and the size (size of pattern arrangement) and 

ty 

^ embedment depth of the patch that are required for 

^ detecting added information Inf can be determined from 

CP the output resolution of the printer and the resolution 

O of the image. 

12 15 Based upon the aforesaid considerations, a method 

fh of changing embedment depth (±c) and the size of the 

patch in accordance with the resolution of the image 
will be proposed. 

An unit using this method will be specifically 
20 described below. 

The output resolution of the printer and the 
resolution of the image are inputted in the pattern 
arrangement determining unit 0110 in FIG. 1, and a 
pattern arrangement suitable for extraction of added 
25 information is outputted. 

As one example, the case is considered where the 
output resolution of the printer is 1200 dpi, and the 
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image is outputted by printing in such a way that the 
length of the longer edge is about 6 inches. Assume 
that there are images with resolution in the range of 
300 ppi to 600 ppi (with the longer edge having 1800 to 
5 3600 pixels) for the image. 

Assume that with the pattern arrangement for use 
in embedment of added information Inf being dependent 
on the resolution of the image, the pattern arrangement 
„ 4901 in FIG. 49A is used in the case where the 

^ 10 resolution of the image is less than 500 ppi, and the 

fjjf pattern arrangement 4903 in FIG. 49B is used in the 

case where the resolution of the image is 500 ppi or 
£H greater. Assume also that the embedment depth can be 

a 

p appropriately determined using (Equation 47-2). 

q 15 The pattern arrangement is appropriately 

?4 determined by the pattern arrangement determining unit 

0110 in this way, and the embodiment position 
determining unit 0103 on the next stage determines the 
position of embedment, based on the size of the pattern 
20 arrangement inputted from the pattern arrangement 
determining unit 0110. Furthermore, the added 
information embedding unit 0104 performs embedment of 
added information Inf in the image in accordance with 
the position of embedment of the pattern arrangement 
25 inputted from the embedment position determining unit 
0103. 
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On the other hand, extraction of added information 
Inf cannot be performed unless the pattern arrangement 
used for embedment is known. Hence, the resolution of 
the image is determined from the scaling rate outputted 
5 by the registration unit 0203, using a pattern 
arrangement determining unit 2001. 

In the case where the output resolution of the 
printer is fixed, the pattern arrangement used for 
embedment can be determined by the pattern arrangement 

10 determining unit 2001 if the resolution of the image is 
known from the scaling rate. 

Thus, even in the case where the patch and pattern 
arrangement can be varied depending on the resolution 
of the image, information obtained from the 

15 registration signal is used, thereby enabling added 
information Inf to be extracted. 

Furthermore, in this embodiment, as for the 
pattern arrangement, an irrotationally symmetric 
pattern arrangement is used. 

20 As will be described in detail in registration 

processing described later, for an impulse signal 
having a symmetric axis, it is generally impossible to 
determine uniquely a rotation angle applied to the 
image because the signal has frequency symmetry. 

25 Thus, in this embodiment, for determining a 

correct rotation angle, the irrotationally symmetric 



- 60 - 



pattern arrangement is used to determine a rotation 
angle applied to the image. 

FIG. 52 shows a state in which a 6x6 pattern 
arrangement 5201 and a pattern arrangement 5202 
5 obtained by rotating the pattern arrangement 5201 are 
made to overlap each other, in this embodiment. 
Reference numeral 5203 in FIG. 52 denotes a positive 
patch area, and reference numeral 5204 denotes a 
negative patch area. Shaded areas 5205 and 5206 in FIG. 

10 52 show portions in which negative or positive symbols 
of the coefficients of the pattern arrangement 5201 and 
the pattern arrangement 5202 are the same. 

FIG. 53 is a graph showing a relationship between 
the rotation angle \\f and the area of shaded areas 5203 

15 and 5204, in this embodiment. This graph shows that in 
the case where a pattern arrangement as shown in FIG. 
52 is used, a maximum confidence distance cannot be 
obtained unless the pattern arrangement used for 
embedment is used to perform extraction by the 

20 patchwork method. 

That is, if an irrotationally symmetric pattern 
arrangement is used, the rotation angle can be uniquely 
determined by obtaining a rotation angle that results 
in a maximum confidence distance. 

25 Here, coefficients for use in the pattern 

arrangement are not limited to constant values ±c, 
considering that coefficients are variable in 
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accordance with image characteristics such as 
aforementioned resolution of an image. So, 
coefficients are not essential components in the 
irrotationally symmetric pattern arrangement. 

The irrotationally symmetric pattern arrangement 
is a "pattern arrangement in which the positive or 
negative symbols of each corresponding elements are not 
wholly the same if for the pattern arrangement and a 
pattern arrangement obtained by rotating the pattern 
arrangement at an arbitrary angle (except for 0° and 
360°) , each elements are corresponded to each other in 
such a manner that they overlap each other at the 
center" . 

Furthermore, in this embodiment, only start bits 
are used for determining a rotation angle 0. Therefore, 
for embedment of added information (usage information) 
other than start bits, a pattern arrangement that is 
not irrotationally symmetric may be used. However, 
information of the pattern arrangement used for 
embedment should be shared by the digital watermark 
embedding apparatus and the digital watermark 
extracting apparatus . 

Start bits and added information will be described 
in detail in next embedment position determination 
processing . 

[3-3. Embedment position determination 
processing] 
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FIG. 11 is a block diagram showing a detailed 
configuration of the embedment position determining 
unit 0103 of this embodiment. The mask creating unit 
1101 shown in FIG. 11 creates a mask for defining the 
position of embedment of each bit information 
constituting added information Inf. The mask is a 
matrix comprising of position information defining a 
method of relative placement of the pattern arrangement 
corresponding to each bit information (See FIG. 9) . 

One example of the mask is shown by 1701 in FIG. 
17A. A coefficient is assigned in the mask on each-by- 
each basis, and each coefficient has equal occurrence 
frequency in the mask. In the case where this mask is 
used, added information Inf of up to 16 bits can be 
embedded . 

Then, the mask referencing unit 1102 reads the 
mask created by the mask creating unit 1101, and 
corresponds each coefficient of the mask to how-manieth 
bit information is each information to determine a 
method of placing the pattern arrangement for embedment 
of each bit information. 

Furthermore, a mask/pattern arrangement 
corresponding unit 1103 deploys in the position of each 
coefficient in the mask the arrangement element of each 
pattern arrangement (for example, 8x8 size) inputted 
from 0110 in the previous stage. That is, each 
coefficient of the mask shown by 1701 in FIG. 17A (one 
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square) is increased in scale by a factor of 8x8 as 
shown by 1703 in FIG. 17B, so that it can be referred 
to as the position of embedment of each pattern 
arrangement . 

5 The added information embedding unit 0104 

described later embeds each bit information using the 
pattern size, referring to an embedment front-end 
coordinate 1702 shown in FIG. 17B. 
f=g Furthermore, in this embodiment, the aforesaid 

?S 10 mask is created each time image data (blue color 

: '% component) is inputted in the mask creating unit 1101. 

"2 Thus, in the case where image data of large size is 

yi inputted, same added information Inf is repeatedly 

embedded several times. 
Q 15 In the aforesaid method, the configuration of the 

P aforesaid mask (arrangement of the coefficient) serves 

as a key when added information Inf is extracted from 
the image. That is, there is an advantage that only a 
possessor of the key can extract information. 
20 Furthermore, in this embodiment, cases where a 

mask created in advance is stored in an internal 
storing unit of the mask creating unit 1101 or the like 
and it is called as necessary, without creating the 
mask in real time, are included in the category. In 
25 this case, quick shift toward following processing is 
possible. 
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Details of each processing performed in the 
embedment position determining unit 0103 will be now 
described . 

[3-3-1. Mask creating unit] 
5 First, the mask creating unit 1101 will be 

described . 

In the case where in embedment of added 
information Inf using the patchwork method, information 
±0 is embedded with a significant process applied to the 

fy 10 pixel value in order to enhance resistance to attacks 

Sj (for example, in the case where the value of c of the 

rfi pattern arrangement is defined to be large) , 

f«3 degradation of image quality is less noticeable for a 

?Z so called edge portion with a rapid change in pixel 

^ 15 values in the image expressed by original image data, 

?== but for the plane portion with less change in pixel 

values, the portion in which the pixel value is 
processed is noticeable as noise. 

Spatial frequency properties perceived by human 
20 eyes are shown in FIG. 13. The lateral axis represents 
the spatial frequency (Radial frequency) , and the 
vertical axis represents the response value of visual 
senses. It is understood from FIG. 13 that when 
information is embedded with pixel values manipulated, 
25 degradation of image quality is noticeable in the low 
frequency domain that can be sensitively perceived by 
human eyes. 
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Thus, in this embodiment, considering properties 
of blue noise masks and cone masks that are usually- 
used in processing of making multi-valued images to be 
binary ones, arrangement of the pattern corresponding 
5 to each bit is performed. 

Now, properties of blue noise masks and cone masks 
will be briefly described. First, the property of the 
blue noise mask will be described. The blue noise mask 
has the property of developing into a blue noise 

10 pattern irrespective of the threshold value with which 
binary formation is performed. This blue noise pattern 
is a pattern having a frequency property of the spatial 
frequency being heavily distributed in the high 
frequency domain. 

15 Part of a blue noise mask is shown in FIG. 37. 

Also, a schematic diagram of the spatial frequency 
property of the blue noise mask made to be binary with 
a threshold of 10 is shown in 1401 of FIG. 14. 

The lateral axis of 1401 represents the radial 

20 frequency, which shows a distance from a origin 

(direct-flow component) when the blue noise mask is 
subjected to Fourier transformation. The vertical axis 
represents a power spectrum, which is a value 
determined by calculating the sum of the square of an 

25 amplitude component positioned at the distance 

indicated by the lateral axis (radial frequency) and 
then obtaining the average value thereof. Furthermore, 
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this figure shows a two-dimensional frequency property 
of an image with one-dimensional graphical 
representation to aid in visual understandings. 

Making a comparison with FIG. 13, it is understood 
that the blue noise mask has high frequency components 
heavily distributed, and thus is less perceptive to 
human eyes. Therefore, for an inkjet printer and the 
like, blue noise is used when a gray scale of a multi- 
valued image is expressed by an area gray scale using 
dots, whereby spatial frequency components can be 
heavily distributed in the high frequency domain to 
express the area gray scale without being perceived 
prominently by human eyes. 

One example of processes of generating the blue 
noise mask will be shown below. 

1. A white noise is generated. 

2. A binary image P g i of a gray scale g (the initial 
value is a white noise mask) is subjected to low pass 
filtering to generate a multi-valued image P g i. 

3. The image of the gray scale g (initial value: 127) 
is compared with the low pass filtering image P g i 
(multi-valued) , and the black-and-white image of the 
binary image Pg is reversed in decreasing order of 
error to obtain a binary image P g i+i . 

4. The processes 2 and 3 are repeated until the error 
reaches a minimum, the binary image P g i(the initial 
value is a white noise mask) is gradually changed to 



- 67 - 



the binary image P g (blue noise mask) of the gray scale 
g (initial value: 127). 

5. Binary black (white) points of the gray scale 
g+l(g-l) are given to the Pg image on random positions. 
The aforesaid processes 2, 3 are repeated to obtain P g+ i 

(Pg-l) - 

The aforesaid processes are repeated, whereby blue 
noise masks at all gray scales are created to generate 
a dither matrix. 

For example, for 32x32 blue noise masks, four 
points are increased (decreased) for each gray scale. 
At this time, however, since the black (white) bit 
determined at the previous gray scale g cannot be 
reverted so as to give 256 gray scales, limiting 
conditions are stricter in low or high gray scales, 
thus rising a disadvantage that only random patterns of 
poor uniformity can be obtained. 

The occurrence frequency distribution (histogram) 
1201 of each coefficient constituting the blue noise 
mask is shown In FIG. 12. In FIG. 12, all values 
(coefficients) of 0 to 255 exist in the mask in equal 
numbers . 

The aforesaid techniques of using the blue noise 
mask for making multi-valued images to be binary are 
well known, and is disclosed in detail in, for example, 
"J. Opt. Soc. Am A/Vol. 9, No. 11/November 1992 Digital 
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halftoning technique using a blue-noise mask Tehophano 
Mitsa, Kevin J. Parker" . 

Properties of cone masks will be now described. 
For one thing, the cone mask is characterized in that 
5 when each coefficient included in this mask is made to 
be binary, periodic or pseudo periodic peaks are 
produced on the spatial frequency domain expressing 
binary information obtained at this time, as shown by 
1402 in FIG, 14. However, it is designed so as not to 

10 produce peaks in the low frequency domain. 

Part of the coefficient arrangement of a cone mask 
is shown in FIG. 38. No peaks are produced in the low 
frequency domain irrespective of the threshold with 
which the cone mask is made to be binary, because 

15 appropriate distances are maintained among dots. 

The outline of the spatial frequency property when 
the cone mask is made to binary with the threshold of 
10 of the cone mask is shown by 1402 in FIG. 14. It is 
understood that for the property of 1402, low frequency 

20 components are few as in the case of the spatial 
frequency property of the blue noise mask. 

In the case of the cone mask, high-density areas 
in the embedment position are few compared to the blue 
noise mask because peaks are produced at frequencies 

25 higher than low frequencies that the blue noise mask 

has, whether the threshold value is low or high. Thus, 
there is an advantage in that embedment noise occurring 
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when added information Inf is embedded is further less 
noticeable compared to the blue noise. 

Also, the frequency with which coefficients 
constituting the cone mask takes on the occurrence 
5 frequency (histogram) shown by 1201 of FIG. 12, as in 
the case of the blue noise mask. 

Thus, if the pattern corresponding to each bit 
information constituting added information Inf is 
embedded in image data in correspondence with the 
10 coefficient of this mask, the pattern corresponding to 
each bit information can be placed in this image data 
in equal numbers, resulting in balanced embedment of 
added information Inf. 

In this embodiment, the cone mask is used for 
15 embedment reference masks because of the advantages 
described above. 

[3-3-2. Mask referencing unit] 

The mask (cone mask) created by the mask creating 
unit 1101 is inputted by the mask referencing unit 1102. 

20 The mask referencing unit 1102 corresponds the position 
of embedding N bit information to be embedded in the 
image to the mask number (pixel value) to determine the 
embedment position. 

A method of determination of embedment positions 

25 performed by the mask referencing unit 1102 in this 

embodiment will be described below. Although the above 
described mask is used in this embodiment, explanation 
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is presented here using a 4x4 mask shown by 1501 in FIG. 
15 for making the explanation understood easily. 

The mask in FIG. 15 has 4x4 coefficients, and has 
coefficient values of 0 to 15 placed on an one-by-one 
5 basis. The position of embedment of added information 
Inf is referred to using this 4x4 mask. In the case of 
the mask for use in this explanation, added information 
Inf of up to 16 bits can be embedded, but the case 

\Q where added information Inf of 8 bits is embedded will 

fU 10 be described below. 

a [as 

%j First, the configuration of added information Inf 

fP will be described using FIG. 36. As shown in this 

r«t figure, added information Inf is comprised of a start 

m bit Infi and usage information Inf2. This start bit 

^ 15 Infi is used by an offset matching unit 2003 included 

^ in the watermark embedding apparatus for recognizing 

that the position where actual added information Inf is 
embedded is displaced from the ideal position and 
correcting in accordance therewith the position of 
20 start of embedment of the watermark (added information 
Inf) . Details thereof will be described later. 

Also, the usage information Inf 2 is original added 
information, namely information that is actually used 
as additional information of image data I. If the 
25 purpose is to track causes when image data wl is 
illegally accessed, for example, the ID of the 
apparatus shown in FIG. 1, the ID of the user or the 
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like is included in this information. Also, if the 
purpose is to impose a ban on the copy of printed image 
data wl, control information showing that copy is 
prohibited is included . 
5 In this embodiment, the start bit is of 5 bits, 

and a bit string of "11111" is used. However, the 
present invention is not limited thereto, and bit 
numbers other than 5 bits of added information Inf can 
be used as the start bit, and in a similar way, bit 

10 strings other than the bit string of "11111" can be 

used. However, the number of bits of the start bit and 
the bit string are necessarily shared by the digital 
watermark embedding apparatus and the digital watermark 
extracting apparatus . 

15 A simple example in which a cone mask constituted 

by 4x4 coefficients as described above is used to embed 
total 8 bits of added information Inf constituted by 5 
bits of start bit and 3 bits of usage information will 
be described . 

20 However, the present invention is not limited 

thereto. It can also be applied, for example, to the 
case where a 32x32 cone mask is used to embed total 69 
bits of added information Inf constituted by 5 bits of 
start bit and 64 bits of usage information. 

25 Assume that added information Inf has 5 bits 

"11111" of start bit and 3 bits "010" of usage 
information. Each has bit information in which the 
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first, second, third, fourth, fifth, sixth, seventh and 
eighth bits are 1, 1, 1, 1, 1, 0, 1 and 0, respectively. 

Each of patterns corresponding to these respective 
bits (See FIG. 9) is assigned to a position 
5 corresponding to each of coefficients of the cone mask, 
and each pixel value of original image data is changed 
by ±c in accordance with this positional relationship. 
By this, one added information Inf is embedded in the 
original image data that is the equivalent in size of 

10 one cone mask. 

In this embodiment, a threshold value is 
determined on the basis of the number of minimum bits 
required for embedding added information Inf, and each 
corresponding bit information is embedded in the 

15 position where a coefficient smaller than or equal to 
this threshold value is placed. By this, one added 
information Inf is embedded in one cone mask, 
irrespective of the number of bits of added information 
Inf. 

20 Furthermore, the present invention is not limited 

to the aforesaid method, and arrangements may be made 
such that each corresponding bit information is 
embedded in the position where a coefficient larger 
than or equal to a certain threshold value is placed, 

25 and based on this, a threshold value is determined- 
Then, in this embodiment, the ratio of the number 
of coefficients smaller than or equal to a threshold 
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value for use in embedment to the number of 
coefficients of the entire mask is called a fill factor, 

For correctly embedding 8 bits added information 
Inf over an integer number of times, a threshold value 
for determining which coefficient is used for the 
embedment reference position in the mask 1501 shown in 
FIG. 15 should be 8 or 16. For this threshold value, 
an optimal value is determined, considering influence 
on resistance and image quality. 

At this time, in the case where the threshold 
value of the mask is defined as 8, the fill factor 
equals 50 %. That is, 50 % of the original image data 
referred to the aforesaid mask are object for 
processing using the pattern arrangement in FIG. 9. 

One example of correspondence relationship between 
each bit and the coefficient in the mask is shown in 
Correspondence Table 1. 

[Correspondence Table 1] 



Order of bit information to 
be embedded 



SI 



S2 



S3 



S4 



S5 



1 



Coefficients in the mask 



20 Here, SI to S5 denote bit information (start bit) 

for registration that is used by an offset matching 
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processor. Three bits of 1 to 3 denote usage 
information . 

According to the correspondence in Correspondence 
Table 1, each bit information is embedded in the 
5 position of input image data corresponding to the 

position of the coefficient (0 to 7) expressed by 1601 
in FIG. 16, using the pattern (See FIG. 9) . The 
correspondence relationship between the order of bit 
information to be embedded and the coefficient in the 

10 mask is part of key information, and each bit 
information cannot be extracted unless this 
correspondence relationship is known. In this 
embodiment, to simplify explanation, SI to S5 and 3 
bits of information correspond to coefficients of 0 to 

15 the threshold value, one by one, , as shown in 
Correspondence Table 1. 

Now, the fill factor when a cone mask of 32x32 
size is actually used to perform embedment will be 
somewhat described. Furthermore, the processing 

20 procedure is same as that in the case where the mask 
1501 is used. 

First, a threshold value required for embedding 
correctly added information Inf over an integer number 
of times is determined, considering degradation of 

25 image quality during embedment. Furthermore, the 

number of coefficients smaller than or equal to the 
threshold value is divided by the number of bits 
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constituting added information Inf N to determine how 
many times each bit can be embedded with one mask size, 
so that each bit constituting the added information Inf 
is embedded repeatedly over equal numbers of times. 
5 For example, in the case where 69 bits of added 

information Inf constituted by 5 bits of start bit and 
64 bits of usage information in one example described 
above is embedded in original image data corresponding 

U3 to coefficient values of 0 to 255, the threshold value 

fO 10 is defined as, for example, 137. 

%j In this case, the number of effective coefficient 

rh values in the mask equals 138. Since the number of 

dots required for expressing one added information Inf 
IZ is 69, each bit information can be embedded two times 

15 (138/69 = 2) for one mask size. 
^ Furthermore, the reason for performing embedment 

for all the points having coefficients smaller than or 
equal to a certain threshold value when the embedment 
position is determined using the cone mask lies in 
20 taking advantage of the property of the cone mask 

producing no peaks in low frequency components of the 
spatial frequency. 

In the case where the embedment position is 
determined as described above, and as a consequence, 
25 the embedment fill factor and the amount of embedment 
information are defined as 50 % and 69 bits, 
respectively, the relationship between each bit 
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information constituting added information Inf and each 
coefficient value constituting the cone mask is as 
shown in Correspondence Table 2 . 
[Correspondence Table 2] 



The order of each bit 
information 



Coefficient values in the 
cone mask 



SI 



0,1 



S2 



2,3 



S3 



4,5 



S4 



6,7 



S5 



8,9 



10,11 



12,13 



64 



136,137 



Here, SI to S5 denote start bits, which are bit 
information for registration that is used by the offset 
matching processing unit. 1 to 64 denote usage 
information. 

10 However, the present invention is not limited 

thereto, and other correspondence relationship between 
each bit information and each coefficient value may- 
adopted as long as the pattern shown in FIG. 9 is used 
to embed each bit information one after another in all 

15 the positions of coefficients of 0 to the threshold (or 
from the threshold to 255) . 

In the case of the 32x32 cone mask, there are four 
positions having the same coefficients in one mask, 
respectively . 
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In the case where each bit information is embedded 
in original image data based on the above described 
Correspondence Table 2 for all coefficients, each bit 
information constituting added information Inf is 
5 embedded over almost equal numbers of times if a cone 
mask of large size such as 32x32 or 64x64 is used. 
Also, single bit information is spread and embedded in 
original image data. 

In the patchwork method, the embedment position is 
10 conventionally selected at random, but in this 
embodiment, a similar effect can be obtained by 
referring to the aforesaid cone mask and degradation of 
image quality is reduced. 

As a consequence, the mask referencing unit 1102 
15 obtains the coordinate (x, y) of a position 
corresponding to each bit information. 

When the information is expressed by: 

Arrangement S[bit] [num] = (x, y) , 
the bit denotes start bits SI to S5 and 1 to 3 bits of 
20 usage information, in the case of Correspondence Table 
1. Also, the num represents the order assigned to each 
coefficient appearing repeatedly in the cone mask. The 
relative coordinate in the mask is given to the (x, y) . 

The above described operations are performed by 
25 the mask referencing unit 1102. 

[3-3-3. Mask/Pattern arrangement corresponding 

unit] 
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The position of embedment in the cone mask of each 
bit information obtained by the mask referencing unit 
1102 is inputted in the mask/pattern arrangement 
corresponding unit 1103. 
5 Because the embedment position determined by the 

mask referencing unit 1102 is the position of a pattern 
of each bit information (equal to 8x8 pixels), addition 
areas (+c) , subtraction areas (-c) and others (0) shown 
Tp in FIG. 9 should be further assigned in the case of the 

rn 10 patchwork method. Hence, the mask/pattern arrangement 

s Is? 

^ corresponding unit 1103 performs operations of 

^ deploying the pattern arrangement of 8x8 size 

I corresponding to FIG. 9 in all the positions of the 

: 

ts:=f 

=H cone mask referred to by the mask referencing unit 1102. 

15 Specifically, the arrangement S[bit] [num] = (x, y) 

M= obtained by the mask referencing unit 1102 is 

multiplied by the size in the lateral direction of 
pattern arrangement for the x coordinate, and is 
multiplied by the size in the vertical direction of 
20 pattern arrangement for the y coordinate. As a 

consequence thereof, the coordinate 1701 in the mask of 
FIG. 17A becomes a front coordinate 1702 of FIG. 17B 
with one pixel in the mask scaled up to one pattern 
arrangement . 

25 When the pattern arrangement shown in FIG. 19 is 

used from this front coordinate, embedment can be 
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performed without causing superimposition over the area 
1703 having a size of pattern arrangement. 

The coordinate (x, y) changes to a coordinate (x 1 , 
y'), but the bit and num of the arrangement S[bit][num] 
5 does not change. Therefore, it is possible to define 
the (x ? , y f ) as a front position in which the pattern 
arrangement and embed a plurality of bit information as 
added information Inf corresponding the bit of the 
C3 arrangement S[bit] [num] . 

tQ 10 Furthermore, a large mask in which each 

"2 coefficient of the cone mask is developed to (scaled up 

2 to) a 8x8 pattern arrangement by the mask/pattern 

• M arrangement corresponding unit 1103 is called scale-up 

4f ma s k . 

□ 15 The size of the aforesaid scale-up mask equals a 

□ size of ( 32x8 ) x ( 32x8 ) , this size is the equivalent of a 
minimum image unit (referred to as a macro block) 
required for embedding at least one added information 
Inf. 

20 Furthermore, in this embodiment, the size of the 

pattern arrangement is not limited to 8x8 as described 
for the pattern arrangement determining unit 0110. In 
the present invention, the sizes of the pattern 
arrangement and the patch are selected and determined 

25 (optimized) by the pattern arrangement determining unit 
0103, in accordance with the resolution of the image or 
the output resolution of the printer. Even when the 
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size of the pattern arrangement does not equal 8x8, the 
mask/pattern arrangement corresponding unit 1103 can 
use a similar unit to determine the position of 
embedment of added information Inf. 
5 Operations performed by the mask/pattern 

arrangement corresponding unit 1103 have been described 
above . 

In the case of small masks, there is less degree 
of freedom in the position of placement of dots during 
Iz. 10 creation and it is more difficult to create masks 

5 5? having desired properties such as cone masks, compared 

2 to the case of large masks. For example, in the case 

E- : where a small mask is repeatedly assigned to all image 

C3 data to embed added information Inf, the spatial 

C3 15 frequency that the small mask has appears on the all 

p image data. 

On the other hand, since completed added 
information Inf is extracted from one mask, the size of 
the mask is set to be large, resulting in reduced 
20 resistance to clipping (possibility that added 

information Inf can be extracted from partial image 
data wl ' ) . Hence, it is necessary that the size of the 
mask should be determined, considering the balance 
between the aforesaid resistance to clipping and 
25 degradation of image quality. 
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Operations performed by the embedment position 
determining unit 0103 in FIG. 1 have been described 
above . 

[3-4. Added information embedment processing] 
5 The added information embedding unit 0104 in FIG. 

1 actually embeds added information Inf, referring to 
the position of embedment of each bit information in 
image data, which is determined as described above. An 
operational flow of processing of embedding repeatedly 
10 added information Inf in this embodiment is shown in 
FIG. 10. 

In a system shown in FIG. 10, a plurality of 
assignable macro blocks is assigned to the whole image, 
and bit information of the first bit is first embedded 

15 repeatedly in these macro blocks, followed by embedding 
repeatedly the second bit, third bit . . . , one after 
another. This is comprised of procedures of subjecting 
all unprocessed macro blocks to embedment processing of 
1001 to 1003 if there is bit information that is not 

20 yet subjected to embedment processing. 

However, this embodiment in this embodiment is not 
limited this order, and internal-external relationship 
of two loops may be inverted. That is, if there are 
unprocessed macro blocks, a change may be made to a 

25 procedure of embedding in those macro blocks all the 
bit information that is not yet embedded. 
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Specifically, embedment of added information Inf 
adds the pattern arrangement in FIG. 9 if each bit 
information to be embedded is "1", Also, if the bit to 
be embedded is "0", the pattern arrangement in FIG. 9 
5 is subtracted, namely those in FIG. 9 with positive and 
negative symbols reverted are added. 

The aforesaid addition and subtraction processing 
is achieved by switching and controlling the switching 
unit 1001 in FIG. 10 in accordance with bit information 

10 to be embedded. That is, it is connected to the adding 
unit 1002 when bit information is "1", and it is 
connected to the subtracting unit 1003 when the 
aforesaid bit information is "0". The processing of 
1001 to 1003 is performed referring to bit information 

15 and information of pattern arrangement. 

Now, one of the aforesaid bit information being 
embedded is shown in FIG. 19. This figure shows an 
example in which the bit information to be embedded is 
"1", namely the pattern arrangement is added. 

20 In the example shown in FIG. 19, I(x, y) denotes 

the original image, and P(x, y) denotes the 8x8 pattern 
arrangement. Each coefficient constituting the 8x8 
pattern arrangement is superimposed over the original 
image data (blue color component) identical in size to 

25 this pattern arrangement, and values in the same 

position are subjected to addition and subtraction with 
each other. As a consequence thereof, I' (x, y) is 
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calculated, and is outputted to the color component 
synthesizing unit 0105 in FIG. 1 as image data of the 
blue color component in which bit information is 
embedded. 

5 The aforesaid addition-and-subtraction processing 

using the 8x8 pattern arrangement is performed 
repeatedly for all the embedment positions (positions 
to which the pattern arrangement for embedding each bit 
information is assigned) determined with the above 

10 described Correspondence Table 2 . 

Furthermore, in this embodiment, the size of the 
pattern arrangement is not limited to 8x8, as described 
in the explanation for the pattern arrangement 
determining unit 0110. In the present invention, the 

15 sizes of the pattern arrangement and the patch are 
selected and determined (optimized) by the pattern 
arrangement determining unit 0110, in accordance with 
the resolution of the image or the output resolution of 
the printer. Even when the size of the pattern 

20 arrangement does not equal 8x8, the added information 
embedding unit 0104 can use a similar unit to embed 
added information Inf. 

Now, loop processing in FIG. 10 being performed is 
shown in FIG. 18A and B. 

25 In FIG. 18A, for embedding each bit information 

repeatedly, the macro block 1802 is assigned repeatedly 
to the entire image 1801 (1803) in the raster order 
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from top left to bottom right to perform embedment 
(1001 to 1003 in FIG. 10), as shown in FIG. 18B. 

The above operations are performed by the added 
information embedding unit 0104, and added information 
5 Inf is embedded in the whole image. 

By the aforesaid processing, added information Inf 
is embedded in image data. If each pixel of the image 
data in which this added information Inf is embedded is 
expressed by a sufficiently small number of dots, the 
10 size of the pattern arrangement is sufficiently small, 
and thus each of these pattern arrangements is 
perceived as very small points by human eyes. Thus, 
the spatial frequency property of the cone mask is 
maintained, thus making it difficult to be seen by 
15 human eyes. 

[3-5. Compression and expansion of files] 
In this embodiment, added information is embedded 
by the added information embedding unit 0104, followed 
by performing compression of files/storage in 
20 memory/expansion of files. 

Here, a method of designing embedment of digital 
watermarks in which compression of images in which 
digital watermarks are embedded is considered. 
[3-5-1. JPEG compression and coding] 
25 FIG. 39 shows chromaticity spatial frequency 

properties of the visual sense. Each curve is obtained 
by using a spatial sine wave pattern white-black color 
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(monochrome) and red-green colors or yellow-blue colors 
that are a pair of opposite colors equal in intensity 
to each other to measure a limit within which the 
spatial sine wave pattern can be recognized by human 
5 eyes when the period and contrast of the aforesaid 
pattern are changed. 

In FIG. 39, sensitivity reaches the maximum at 
about 3 [period/deg] for white-black color (light-and- 
dark information) , but sensitivity reaches the maximum 

10 at about 0.3 [period/deg] for chromaticity (red-green 
colors and yellow-blue colors) . 

It can be understood therefrom that light-and-dark 
information is sensitive for identification of minute 
areas such as resolution of images and chromaticity has 

15 influence on how spatially wide areas (in which the 

spatial frequency is low) are viewed. Also, it can be 
understood that the yellow-blue pattern is not involved 
in identification of such minute spatial information as 
the red-green pattern is involved in. 

20 From the above, it can be understood that the 

method in which gray scale images having only 
brightness components are directly modulated to embed 
digital watermark information leads to more noticeable 
degradation of image quality than the method in which 

25 digital watermark information is embedded in color 

components of color image data. Also, it can be said 
that for color image data composed of RGB, the method 
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in which digital watermark information is embedded in 
blue color components (B) leads to degradation of image 
quality least perceptible by human eyes. 

Furthermore, when the color component is changed 
5 for embedding digital watermark information in the 
color component, it is noticeable as color 
inconsistencies if a person make a visual observation 
in a spatially broad area (conditions of low spatial 
tj frequencies) , but it is less noticeable than cases 

fn 10 where digital watermark information is embedded in 

f £ s 

brightness, if a person makes a visual observation in a 

'--1 

p spatially narrow area (conditions of high spatial 

tf\ 

"* frequencies) . 

^rj In this embodiment, for a gray scale image having 

W 15 only one element in one pixel, the image is subjected 

P to conversion to color image data having a plurality of 

elements in one pixel before embedment of digital 
watermark information (such as added information Inf) 
is performed, thus bringing about the increased effect 
20 of preventing degradation of image quality, compared to 
cases where digital watermark information is directly 
embedded in a normal gray scale image. 

If cases where digital watermark information is 
embedded in gray scale image data and cases where 
25 digital watermark information is embedded in only one 
kind of elements of two or more kinds of elements 
constituting color image data are compared to each 
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other, the later is more capable of maintaining 
desirable image quality when the image is outputted at 
high resolution (when the value of one pixel is 
expressed in gray scale with a small number of dots). 
5 However, there is also a disadvantage in that if 

simple considerations are given, outputted color image 
data (file size) is three times as large as original 
image data. 

Thus, in this embodiment, for reducing the file 

10 size to a minimum, image data in which digital 

watermark information is embedded is further subjected 
to JPEG compression and coding in the JPEG compressing 
and coding unit 0106. 

Generally, JPEG compression and coding is a 

15 technique of removing components to which the visual 
sense of human beings is not sensitive to reduce the 
amount of data, taking advantage of properties of human 
visual sense. The digital watermark technique, on the 
other hand, is a technique of embedding information in 

20 components to which the visual sense of human beings is 
not sensitive. Therefore, coexistence of the JPEG 
compression and coding technique and the digital 
watermark technique is difficult, and it is believed 
that the JPEG compression and coding is a type of 

25 attacks to digital watermark information. 

A method of giving resistance to JPEG compression 
and coding will be briefly described below. A pattern 
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arrangement as shown in FIG. 9 that is used in this 
embodiment is defined so that added information already 
embedded in color image data is not lost through sub- 
sampling and quantization processing of the color 
5 difference component. 

First, the JPEG compression and coding system will 
be briefly described. Color image data that is 
inputted in the JPEG compressing and coding unit 0106 
^3 is transformed into brightness (Y) and color difference 

Tin 
~~? 

fU 10 (Cr, Cb) . When color image data constituted by 

original red color components (R) , green color 

£f| components (G) and blue color components (B) is 

q inputted, equations of: 

S Y=0 . 2 9900XR+0 . 58 700XG+0 . 114 00XB 

^ 15 Cr=0 . 5000 OXR-O . 418 69XG-0 . 08131XB 

" Cb=-0 . 168 7 4XR-0 . 3312 6XG+0 . 5 0000XB 

are used to subject the original color image data to 
format conversion into different color image data 
constituted by brightness (Y) and color difference (Cr, 
20 Cb) . 

The image data decomposed into the brightness 
components and color-difference component is divided 
into blocks of 8x8 pixels shown in FIG. 40 in the 
raster order from top left of the image. In JPEG 
25 compression and coding, processing of compression and 

coding is performed in repetition for these 8x8 blocks. 
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Processing of sampling of color components in JPEG 
compression and coding will be now described. In JPEG 
compression and coding, sampling of color-difference 
components is performed for every 8x8 pixels in 
5 accordance with the sampling rate of the sampling 
option . 

FIG . 41 shows sampling of image data. A procedure 
of sampling with the rate of 4 : 2 : 2 in JPEG 
compression and coding will be presented below. In FIG. 

10 41, reference numeral 4101 denotes the brightness 

component of 4x4 pixels. For the brightness component, 
because it has a large amount of visually important 
information, 4x4 pixels are directly outputted without 
performing scale-down processing. 

15 Reference numeral 4103 denotes the color 

difference component (Cr, Cb) of 4x4 pixels. For the 
chromaticity component, because of less visual 
sensitivity, it is subjected to scaling-down to reduce 
two pixels to one pixel in the lateral or vertical 

20 direction. As a consequence thereof, the color- 
difference component (Cr, Cb) of 4x4 pixels is 
transformed into that of 4x2 pixels 4104. The sampling 
described above reduces color-difference component of 
8x8 pixels to 8x4 pixels. 

25 Thus, as a consequence of sampling with the ratio 

of 4: 2: 2, the brightness component Y, color- 
difference component Cr and color-difference component 
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Cb of 8x8 pixels become those of 8x8 pixels, 8x4 pixels 
and 8x4 pixels, respectively. Each of the aforesaid 
pixels subjected to sampling is then subjected to DCT 
(discrete cosine transformation) computation, 
5 quantization, zigzag scanning, Huffman coding, and so 
on in accordance with known procedures. 

Also, taking advantage of the fact that the 
property of visual sense of human beings is not so much 
sensitive to high frequency components, the number of 

10 steps of quantizing DCT coefficients to high frequency 
components is reduced, thereby enabling effective 
compression. Also, quantization is performed so that 
generally, the brightness component undergoes less 
steps of quantization than the color-difference 

15 component. 

Pattern . arrangements having resistance to the 
above described compression and coding processing will 
be now discussed. The pattern arrangement in FIG. 9 
described above is shown again in FIG. 42. In FIG. 42, 

20 an area 4201 having positive elements of +c is called a 
positive patch, and an area 4202 having negative 
elements of -c is called a negative patch. At this 
time, each patch has information heavily distributed to 
the low frequency component in the minimum coding unit 

25 4001 constituted by 8x8 pixels shown in FIG. 40, 

thereby making it possible to enhance resistance to 
JPEG compression. However, this embodiment is limited 
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thereto, and various kinds of definitions may be made 
with the minimum coding unit as 16x16 pixels. 

Also, in the case where sampling with the rate of 
4: 1: 1 (reducing the color-difference component by 
5 every other pixel in vertical and lateral directions) 
or 4: 2: 2 (reducing the color-difference component by 
every other pixel in the vertical or lateral direction) 
is performed, resistance to sampling can be enhanced if 
^ each patch is given a size having a width of 2 pixels 

iTi 10 multiplied by an integer number in the vertical and/or 

s y 

*j\ lateral direction depending on sampling. 

That is, (1) for each patch, one heavily 
B distributed to the low frequency in the minimum coding 

=p unit (8x8 pixels) is used. (2) Each patch is given a 

15 size with 2xN (N denotes an integer number) pixels in 

O 

the vertical and/or lateral direction depending on 
sampling methods. 

Furthermore, in order that each patch has low 
frequency components in each area (8x8 pixels) that is 

20 subjected to JPEG compression and coding, the position 
on the image to which the pattern arrangement is 
assigned and each size of the pattern arrangement (8x8 
pixels in FIG. 9) are preferably synchronized with each 
area that is subjected to coding. 

25 That is, (3) the size and embedment position of 

the pattern arrangement are synchronized with the unit 
size for which JPEG compression and coding is performed. 
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If added information Inf is embedded using, for 
example, a pattern as shown in FIG. 9, considering the 
above conditions, digital watermark information (added 
information Inf) can remain in image data even after 
5 JPEG compression and coding, resulting in possible 
resistance to JPEG compression and coding. 

Furthermore, this embodiment includes in its 
category the case where in the color component 
?=* extracting unit 0101, the gray scale (monochrome) image 

fQ 10 is transformed directly into Y (brightness) and Cr and 

5 p Cb (color-difference) components, and modulation of 

2 embedding added information Inf and the like in only 

• :M the Cb component of those components as an digital 

£5 watermark is performed. In this case, no conversion 

C5 15 into brightness and color-difference components should 

fj be performed in the JPEG compressing and coding unit, 

resulting in reduced processing steps. 

Also, the case where in color component extracting 
unit 0101, the gray scale (monochrome) image is 
20 transformed directly into Y (yellow) , M (magenta) , C 
(cyan) and K (black) components suitable for printing 
that is performed in the subsequent stage, and added 
information Inf and the like are embedded in only the Y 
component of those components as an digital watermark 
25 is included in the category. In this case, a step of 
transforming color components just before a printing 
unit can be omitted. 
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That is, in this embodiment, the component for 
performing the aforesaid embedment is not limited to 
the blue, Cb and Y components, and the case where added 
information Inf and the like are embedded in the part 
5 of all components constituting one pixel is also 
included in the category. 

[3-5-2. Storage in memory] 

The encoded data obtained from the aforesaid JPEG 

1-3 

S3 compression and coding is stored in a memory on a 

LU 

fU 10 temporary basis. This encoded data is read out from 

%| the aforesaid memory to the JPEG expansion and decoding 

fh unit 0108 in timing of transmission to an external unit 

f3 or printing by a printer connected to the subsequent 

?Z stage of the apparatus in FIG. 1. 

!E 15 Furthermore, encoded data in the case where gray 

I s 

? ~" scale image data is transformed into color image data 

on a temporary basis, is modulated to the blue color 
component, and is transformed into color image data 
constituted by brightness and color-difference 

20 components to perform JPEG compression as in the case 
of this embodiment causes a more or less increase in 
data in the color difference component, but does not 
lead to a significant increase in memory capacity, 
compared to the amount of encoded data in the case 

25 where original gray scale image data is transformed 
directly into color image data constituted by 
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brightness and color difference components to perform 
JPEG compression and coding. 

That is, provided that digital watermark 
information is embedded in the original image data, and 
5 is then subjected to JPEG compression and coding, a 

method of embedding digital watermark information (such 
as added information Inf) in gray scale image data as 
in the case of this embodiment has an advantage that 
image quality can be improved without significant 

10 increase in general data amounts, compared to methods 
in which normal gray scale image data is directly 
modulated to embed digital watermark information. 
[3-5-3. JPEG expansion and decoding] 
The JPEG expanding and decoding unit 0108 reads 

15 encoded data from the memory 0107 in timing of 

transmission to an external unit or printing by the 
printer 0109, and uses an inverse procedure to the 
aforesaid compression system to decode color image data. 
[3-6. Registration processing] 

20 The registration unit 0203 in FIG. 2 that is 

provided for the digital watermark extracting apparatus 
will be now described in detail. The registration unit 
0203 is a unit positioned at a stage before the added 
information extracting unit 0204, which is involved in 

25 pre-processing for processing of extraction of added 
information Inf. The image of blue color components 
extracted by the color component extracting unit 0202 
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by the previous stage is inputted by the registration 
unit 0203. 

The registration unit 0203 corrects a difference 
in scale between the image data wl outputted from the 
5 digital watermark embedding apparatus and the image 
data wl ' inputted by the digital watermark extracting 
apparatus, and rotation thereof. However, rotation that 
is corrected here represents one of rotation angles 
that possibly enable extraction, and correction for a 

10 precise rotation angle is performed in offset matching 
processing. The offset matching processing will be 
described later in detail. 

Details of the registration unit 0203 in this 
embodiment are shown in FIG. 7. As shown in FIG. 7, 

15 the registration unit 0203 is comprised of a block 

dividing unit 0701, a Fourier transforming unit 0702, 
an impulse extracting unit 0703, an affine 
transformation calculating unit 0704 and an affine 
transformation correction unit 0705. 

20 In the block dividing unit 0701, block divide 

processing as in the case of the registration signal 
embedding unit 0102 (block dividing unit 0401) 
described above is performed. It is usually difficult 
to extract blocks, as in the case of the registration 

25 signal embedding unit 0102 (block dividing unit 0401), 
by this processing. This is because image data wl in 
which digital watermark information is embedded is 
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subjected to processing of print systems, whereby the 
size is changed, rotation is applied, and displacement 
occurs. 

However, there arises no problem even if this 
5 extraction of blocks is somewhat incorrect. This is 
because in the digital watermark embedding apparatus, 
the registration signal is embedded in amplitude 
spectra of image data. The amplitude spectrum has a 
property of not being influenced by displacement in the 

10 spatial domain of image data. Therefore, there arises 
no problem even if, in each of the digital watermark 
embedding apparatus and the electronic water marl 
extracting unit, some amount of displacement is found 
in blocks divided by each block dividing unit in the 

15 spatial domain. 

The block dividing unit 0701 outputs to the 
Fourier transforming unit 0702 the image data subjected 
to block dividing. The Fourier transforming unit 0702 
transforms image data in the spatial domain into image 

20 data in the frequency domain as in the case of the 

above described registration signal embedding unit 0102 
The image data in the frequency domain subjected to 
Fourier transformation is expressed by amplitude and 
phase spectra. Only the amplitude spectrum of these 

25 spectra is inputted by the impulse extracting unit 0703 
On the other hand, the phase spectrum is discarded. 
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• m 

The image data transformed to the frequency domain 
is inputted by the impulse extracting unit 0703. The 
impulse extracting unit 0703 extracts only impulsive 
signals from the image data transformed into the 
5 frequency domain. That is, it extracts 0502, 0503, 
0504 and 0505 in FIG. 5 that are already embedded in 
image data. 

This can be performed using commonly known image 
processing techniques. For example, it can be achieved 

10 by subjecting to threshold value processing the image 
data transformed to the frequency domain. An example 
thereof is shown in FIG. 8 (a) . A state in which the 
amplitude spectrum 0801 inputted by the impulse 
extracting unit 0703 is subjected to threshold value 

15 processing with a threshold value 0802 is shown in FIG. 
8 (a) . Furthermore, the transformed image data in FIG. 
8 is expressed by one-dimensional representation for 
explanation. An appropriate threshold value 0802 is 
selected, thereby enabling impulse signals to be 

20 extracted. However, original image data similar in 
size to impulse signals existing in the low area is 
extracted at the same time. 

A system of this embodiment to solve this problem 
is shown in FIG. 8 (b) . The image data 0801 

25 transformed to the frequency domain is subjected to 
secondary differential processing. This is the 
equivalent of applying Laplacian filters and the like. 
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Image data obtained by subjecting to secondary 
differential processing for the image data 0801 
transformed to the frequency domain is shown by 0803. 
An appropriate threshold value 0804 is selected to 
5 perform threshold value processing for this data 0803, 
whereby the impulse signal can be extracted. 

A little more detailed principles will be 
described as to extraction of this impulse signal, 
using FIG. 26. Furthermore, in this figure, processing 

10 in the registration signal embedment described above is 
also described. 

In the registration signal embedding unit 0102, 
image data 2601 in the spatial domain is transformed 
into image data 2602 in the frequency domain, and an 

15 impulse signal 2603 is added in the frequency domain. 

The image data in the frequency domain in which 
the impulse signal (registration signal) 2603 is added 
is turned back to a signal 2601' in the spatial domain 
by being subjected to inverse frequency conversion. 

20 The image data 2601' turned back to the spatial domain 
should be influenced by addition of the impulse signal, 
but it is hardly perceptible by human eyes, and 
substantially, the image data 2601 and image data 2601' 
looks as if they are same matters. This is because the 

25 impulse signal 2603 added in the frequency domain is 

distributed in small amplitudes over entire image data, 
by inverse Fourier transformation. 
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If an impulse signal like 2603 in FIG. 26, it 
means that image data having certain frequency 
components is added in the spatial domain. If this 
added impulse signal is higher than the frequency that 
5 can be perceived by human beings and the amplitude is 

below the limit for enabling perception by human beings, 
the added impulse signal cannot be seen by human eyes. 
Thus, it can be said that the above described embedment 
of registration signals is itself a type of digital 

10 watermark processing. 

Furthermore, in this embodiment, the registration 
signal 2603 is embedded in image data 2601, and added 
information Inf that should be actually embedded is 
embedded, followed by reconstructing the signal 2601' 

15 in the spatial domain. 

The registration signal embedded as shown in FIG. 
26 is subjected to Fourier transformation again at the 
time of extraction. By this, in the spatial domain, 
the registration signal 2603 once spread over the whole 

20 image data is transformed to the frequency domain and 
reappears as the impulse signal. 

There is a high possibility that the amplitude of 
this impulse is reduced when the image in which digital 
watermark information is embedded is subjected to 

25 attacks such as irreversible compression such as JPEG 
compression. On the other hand, when the image is 
subjected to geometric attacks such as scaling and 
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rotation, the position of this impulse is shifted. In 
any case, appropriate impulse extraction processing as 
described above is performed, whereby it is possible to 
extract the impulse signal, and a change from the 
5 original image data can be predicted. This change is 

corrected, thereby making it possible to create a state 
where added information Inf that is embedded in this 
embodiment can be reliably extracted. 

By the above described processing, the aforesaid 

10 impulse signal is outputted from the impulse extracting 
unit 0703 in FIG. 7 and is inputted by the affine 
transformation calculating unit 0704. The affine 
transformation calculating unit 0704 determines what 
scaling and rotation have been performed, using the 

15 coordinate of the inputted impulse signal. 

Assume that the digital watermark extracting 
apparatus of this embodiment knows which frequency 
component the impulse signal has embedded in, in 
advance . 

20 FIG. 50A shows an amplitude spectrum showing the 

position of the impulse signal embedded in advance in 
the image, and FIG. SOB shows an amplitude spectrum 
showing the position of the detected impulse signal. 
In this case, the scaling rate can be calculated from 

25 the ratio of this frequency of the impulse signal 

embedded in advance to the frequency of the impulse 
detected. For example, assuming that the frequency of 
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the impulse signal embedded in advance is a, and the 
frequency of the impulse signal detected is b, it can 
be understood that scaling with the ratio of a/b is 
performed . 

5 Also, in the case where rotation is applied to the 

image, a rotation angle applied to the image can be 
estimated from the frequency of the extracted impulse 
signal in a similar way. 

FIG. 51A shows an amplitude spectrum showing the 

10 position of the impulse signal embedded in advance in 
the image, and FIG. 51B shows an amplitude spectrum 
showing the position of the impulse signal extracted 
from the image with the digital watermark embedded 
therein, to which rotation is applied. 

15 At this time, assuming that the angle which the 

straight line between the position of the impulse 
signal embedded in advance and the origin point forms 
with the x axis in the positive direction is 9i and the 
angle which the straight line between the position of 

20 the extracted impulse signal and the origin point forms 
with the x axis in the positive direction is 9 2/ a 
rotation angle applied to the image can be known from a 
difference between 9i and 9 2 . For amplitude spectra, it 
is impossible to determine uniquely the rotation angle 

25 9 because the impulse signal has frequency symmetry. 

Therefore, there may be cases where the rotation angle 
9' (a difference between 0i and 9 2 ) obtained by this 
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registration processing is different from the rotation 
angle 9 applied to the image. 

This is a well known property of Fourier 
transformation. By the above described processing, the 
5 scaling rate rotation angle 9' are outputted from the 
scaling rate calculating unit 0704. The precise 
rotation angle 9 applied to the image is determined in 
offset matching processing. 

However, this embodiment is not limited thereto, 

10 and information of the position in which the 

registration signal is embedded (distance from the 
origin point in the amplitude spectrum of the frequency 
space, in the case of rotation) may be received from 
the digital watermark embedding apparatus, as necessary. 

15 For example, cases where this position information 

is received as an encoded signal to perform the 
aforesaid scaling rate calculation processing are also 
included in the category of this embodiment. By so 
doing, only those who know the registration signal can 

20 correctly extract added information Inf. In this case, 
the registration signal can be used as a key for 
extracting added information Inf. 

The scaling rate and rotation angle 9' outputted 
from the scaling rate calculating unit 0704 are 

25 inputted by the affine transformation correction unit 
0705. Image data wli* is also inputted by the scaling 
unit 0705, and affine transformation correction 
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processing is carried out for image data wli' using the 
inputted scaling rate and rotation angle 9*. Bilinear 
interpolation and bicubic interpolation among other 
things are applicable to the affine transformation 
5 correction processing. And, image data wl2' that has 
been subjected to affine transformation correction 
processing is outputted by the affine transformation 
correction unit 0705. 

Furthermore, in this embodiment, the affine 

10 transformation calculating unit 0704 outputs the 

scaling rate to the pattern arrangement calculating 
unit 0204 in FIG. 1, for another purpose. 

One purpose is to scale up or down the image data 
wli 1 to the image data wl 2 1 through scaling processing, 

15 but another purpose is to determine a pattern 

arrangement used for embedment on the basis of this 
outputted scaling rate in the pattern arrangement 
calculating unit 0204 in FIG. 2. 

[3-7. Added information extraction processing] 

20 Operations of the added information extracting 

unit 0204 in 'FIG. 2 for extracting added information 
Inf from the blue color component of image data wl 1 in 
which this added information Inf is embedded by the 
added information embedding unit 0103 in FIG. 1 will be 

25 now described. A flow of processing of extracting 

added information in this embodiment are shown in FIG. 
20. 
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As shown in FIG. 20, in this embodiment, the 
scaling rate is inputted by the pattern arrangement 
determining unit 2001 from the registration processing 
0202 in FIG. 2, and the pattern arrangement used for 
5 embedment is determined. 

[3-7-1. Pattern arrangement determining unit] 
Image data wl 2 subjected to scaling processing is 
inputted by the added information extracting unit 0204. 
At this time, if the pattern arrangement has been 
10 determined in accordance with the resolution of the 

image and the output resolution of the printer by the 
pattern arrangement determining unit 0110 in FIG. 1, 
the pattern arrangement used for embedment cannot be 
determined unless the resolution of the image and the 
15 output resolution of the printer are known. Extraction 
of added information cannot be performed unless the 
pattern arrangement used for embedment is known. 

In this embodiment, a pattern arrangement in case 
of embedding added information Inf is determined from 
20 the scaling rate inputted from the registration unit 

0202, by the pattern arrangement determining unit 2001. 

Assuming that the input resolution of the scanner 
at the time of scanning a printed matter pwl ' is fixed, 
and the output resolution of the printer is known, a 
25 unit for determining a pattern arrangement from the 
scaling rate will be described. 
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A printed matter pwl is scanned at input 
resolution of 600 ppi by the scanner 0201 in FIG. 2, 
and image data wl is obtained. Assume that the output 
resolution of the printer is 1200 dpi, and the input 
5 resolution of the scanner is 600 ppi, at this time. 

The image data wl is inputted in the color 
component extracting unit 0202, which extracts the 
color component and outputs image data wli. This image 
data wli is inputted by the registration unit 0203, and 
10 the scaling rate and the image wl 2 already subjected to 
scaling are outputted to the added information 
extracting unit 0204 from the scaling rate calculating 
unit 0704 by the registration unit 0202. 

The scaling rate inputted by the added information 
15 extracting unit 0204 is inputted by the pattern 

arrangement determining unit 2001. As an example, an 
image wl in which an digital watermark having a scaling 
rate of 0.80 is embedded will be considered. 

From the scaling rate and the input resolution of 
20 the scanner, it can be understood that the image has 

been outputted from the printer at image resolution of 
600 ppix0.80 = 480 ppi. 

Now, assuming that the resolution of the image of 
500 ppi is a threshold, embedment is performed using 
25 4901 in FIG. 49A (pattern arrangement of 8x8) for a 

pattern arrangement, in the case resolution lower than 
500 ppi. In the case of resolution higher than 500 ppi, 
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on the other hand, if the correspondence relationship 
where embedment is performed using 4 903 in FIG. 4 9B 
(pattern arrangement of 12x12) for a pattern 
arrangement is specified considering gray scale 
5 conversion, it can be understood that for the image wl 
in which the digital watermark having a scaling rate of 
0.80, embedment is performed using the pattern 
arrangement 4901 in FIG. 49A (pattern arrangement of 
„ 8x8) . 

^ 10 As described above, the pattern arrangement 

determining unit 2001 determines the pattern 
*2 arrangement used for embedment of added information Inf 

CH based on the scaling rate, and outputs the embedment 

p position determining unit 2002 by the subsequent stage, 

fg 15 Furthermore, the orthogonal pattern arrangement 

f« 4902 in FIG. 49C corresponding to the pattern 

arrangement 4901 in FIG. 49A, and the orthogonal 
pattern arrangement 4904 in FIG. 49D corresponding to 
the pattern arrangement 4903 in FIG. 49B are used in an 
20 extracting unit 2005 according to the second pattern 
arrangement in FIG. 20. Orthogonal pattern 
arrangements will be described later in detail. 

[3-7-2. Embedment position determination 
processing] 

25 In the embedment position determining unit 2002, 

which area of image data wl2* (blue color component) 
added information Inf is extracted from is determined. 
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Operations carried out by this embedment position 
determining unit 2002 are same as those by the 
embedment position determining unit 0103, and for this 
reason, areas determined by 0103 and 2002 are the same. 
5 The aforesaid Correspondence Table 2 and further 

the pattern arrangement shown in FIG. 9 are used to 
extract added information Inf from a determined area. 
Here, embedment of added information Inf is achieved by 
convoluting the pattern arrangement against the 

10 determined area. 

Furthermore, in the case where the pattern 
arrangement is variable in response to the output 
resolution of the image, the pattern arrangement 
inputted from the pattern arrangement determining unit 

15 2001 in FIG. 20 shall be used. Hereinafter, 

explanation will be presented only on the case where 
the pattern arrangement inputted from the pattern 
arrangement determining unit 2001 in FIG. 20 is 8x8, 
but even in the case of other pattern arrangements, 

20 similar operations will be carried out. 

[3-7-3. Confidence distance computing unit] 
The confidence distance d is a calculated value 
required when embedded information is extracted. A 
method of determining the confidence distance d 

25 corresponding to each bit information is shown in FIG. 
6. 
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First, processing performed by a convolution 
computing unit 0601 will be described using FIG. 21 and 
FIG. 22. Examples of embedment of one bit of 
information constituting added information Inf are 
5 shown in FIGS. 21 and 22. 

FIG. 21 shows an example of performing processing 
of extracting 1 bit information for image data (blue 
color component) I" (x, y) in which this 1 bit 
C3 information constituting added information Inf is 

W 10 embedded, and FIG. 22 shows an example of trying to 

perform processing of extracting 1 bit information for 
image data I" (x, y) in which the above described 1 bit 
information is not embedded. 

In FIG. 21, I" (x, y) is image data in which 1 bit 
™ 15 information is embedded, and P(x, y) is a 8x8 pattern 

— p5 

» 

arrangement (pattern arrangement for extracting added 
information Inf) for use in convolution processing. 
Each element (0, ±c) constituting this pattern 
arrangement is multiplied by a pixel value placed in 

20 the same position in input image data I" (x, y) , and the 
sum of each multiplied value is calculated. In other 
words, P(x, y) is convoluted against I" (x, y) . 

Here, I" (x, y) is a representation including the 
image in the case of the image data I'(x, y) attacked. 

25 If it is not attacked, I" (x, y) = I'(x, y) holds. In 
the case of the image in which 1 bit information is 
embedded in I" (x, y) , there is a very high possibility 
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that a non-zero value is obtained as a result of 
convolution, as shown in FIG. 21. Especially when I" (x, 
y) = I' (x, y) holds, 32 c 2 is obtained as a result of 
convolution . 

Furthermore, in this embodiment, the pattern 
arrangement for use in embedment is same as the pattern 
arrangement for use in extraction. However, this 
embodiment should not be limited to the above described 
example. Generally, in the case where the pattern 
arrangement for use in embedment is defined as P(x, y) , 
and the pattern arrangement for use in extraction is 
defined as P(x, y) , alteration to relationship of P(x, 
y) = aP(x, y) can be made. Here, a represents any real 
number, and in this embodiment, the case of a = 1 will 
be described for simplification. 

In the example shown in FIG. 22, on the other hand, 
computation similar to that described above is 
performed for the image data I" (x, y) in which 1 bit 
information is not embedded. A zero value is obtained 
as a result of convolution computation from the 
original image (corresponding to image data I), as 
shown in FIG. 22. 

A method of extracting 1 bit information has been 
described above, using FIGS. 21 and 22. However, the 
above description refers to the case where a zero value 
is obtained as a result of convolution computation in 
image data I in which added information Inf is to be 
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embedded, which is an ideal case. On the other hand, 
for an area corresponding to the 8x8 pattern 
arrangement in actual image data I, a zero value is 
hardly obtained as a result of convolution computation. 
5 In other words, if for the area corresponding to 

the 8x8 pattern arrangement in the original image 
(image data I), convolution computation is performed 
using the pattern arrangement in FIG. 9 (See also the 
cone mask as placement information) , a non- zero value 
5:7; 10 may be obtained unlike an ideal case. 

4= In a different way, there may be cases where 

=F convolution computation is performed in a same- manner 

3 for the area corresponding to the 8x8 pattern 

J5 arrangement in the image (image data wl) in which added 

5 r 15 information Inf is embedded, which results in not 

12 "32c 2 " but "0". However, each of bit information 

constituting added information Inf is usually embedded 
in the original image several times. That is, added 
information Inf is embedded in the image several times. 
20 Thus, the convolution computing unit 0601 

calculates two or more of sums of results of 
convolution computation, for each bit information 
constituting added information Inf. For example, if 
added information is of 8 bits, eight sums are obtained. 
25 These sums, corresponding to each bit information 

are inputted by an average calculating unit 0602, and 
each thereof is divided to be averaged by the number n 
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of repetitions of the pattern arrangement corresponding 
to each bit information in the whole macro block. This 
averaged value is the confidence distance d. That is, 
this confidence distance d is a value generated by 
5 determining based on which of "32c 2 " and "0" it is more 
similar to. 

However, in the previous explanation for the 
patchwork method, since the confidence distance d is 
f =s defined as d = (l/N)Z(ai - bi) , the confidence distance 

10 d is in the strict sense an average value obtained as a 
result of performing convolution computation using P' 
^ (x, y) = (l/c)P(x, y) . However, even if convolution 

Sm computation is performed using P T (x, y) = a*P(x, y) , 

□ the average value obtained as a result of convolution 

O 15 computation is merely a value of the above described 

« confidence distance d multiplied by a real number, and 

thus a similar effect can be obtained in essence. 

Therefore, in this embodiment, it is utterly 
possible to use for the confidence distance d the 
20 average value obtained as a result of convolution 
computation using P 1 (x, y) = a*P(x, y) . 

The confidence value d obtained in this way is 
accumulated in the storage medium of 0603. 

The convolution computing unit 0601 generates the 
25 above described confidence distance d in repetition for 
each bit constituting added information Inf, and stores 
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the confidence distance d in the storage medium 0603 in 
succession . 

A little more detailed explanation will be 
presented as to this computed value. The confidence 
5 distance d calculated using the pattern arrangement in 
FIG. 9 (See also the cone mask as placement 
information) for the original image data I ideally 
equals 0. However, as is often the case with actual 
image data I, this value is a non-zero value although 
10 it is very close to zero. Examination of the frequency 
distribution of the confidence distance d produced for 
each bit information results in distribution as shown 
in FIG. 23. 

In FIG. 23, the lateral axis represents the value 
15 of confidence distance d produced for each bit 

information, and the vertical axis represents the 
number of bit information for which convolution 
producing the confidence distance d has been performed 
(occurrence frequency of the confidence distance d) . 
20 As is apparent from the figure, it is similar to normal 
distribution. Also, in the original image data I, the 
confidence distance d does not necessarily equal 0, but 
the average thereof equals 0 (or a value that is very 
close to 0 ) . 

25 On the other hand, if the above described 

convolution of not the original image data I but image 
data (blue color component) after bit information "1" 
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is embedded therein is performed against i' (x, y) as 
shown in FIG. 19, the confidence distance d takes on 
frequency distribution as shown in FIG. 24. That is, 
it is shifted in the right direction while keeping a 
5 distribution pattern in FIG. 23 as shown in the figure. 
In this way, for image data after certain 1 bit 
information constituting added information Inf, the 
confidence distance d does not necessarily equal c, but 

r=| the average thereof equals c (or a value that is very 

1% 10 close to c) . 

*U Furthermore, in FIG. 24, an example of embedding 

2 bit information "1" is shown, but in the case of 

y - embedding bit information "0", the frequency 

distribution shown in FIG. 23 is shifted in the left 
C3 15 direction. 

Q As described above, in the case where added 

information Inf (each bit information) is embedded 
using the patchwork method, larger the number of bits 
to be embedded (the number of times the pattern 

20 arrangement is used) , the more likely the statistical 
distribution as shown in FIGS. 23 and 24 is correctly 
produced. In other words, accuracy with which whether 
or not each bit information constituting added 
information Inf is embedded, or whether embedded bit 

25 information is "1" or "0" can be detected is enhanced. 
[3-7-4. Offset matching processing] 
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The offset matching unit 2003 will be now 
illustrated. Image data wl 2 ' after being subjected to 
appropriate scaling is inputted in the offset matching 
unit 2003. After that, start bits are detected using 
5 the computation of confidence distance in FIG. 6. 

Furthermore, as described for registration 
processing, rotation of the image data wl2* inputted is 
not necessarily corrected to the original state. 
[□ Therefore, in this offset matching processing, not 

fn 10 only correction of offset but also processing of 

*J= determining uniquely the rotation angle applied to the 

p image is performed. 

- = The offset matching unit 2003 generates only five 

™ confidence distances corresponding to 5 bits of start 

H 15 bits Infi. As shown in FIG. 36, the start bit Infi is 

part of added information Inf embedded in advance by 
the added information embedding unit 0104, and is the 
equivalent of 5 bits in the case of this embodiment. 
These start bits Infi are the equivalent of 
20 initial 5 bits conceptually, but exist not contiguously 
and densely but rather scatteringly in the image in 
which added information Inf is embedded. This is 
because they are corresponded to respective coefficient 
values constituting the cone mask of the Correspondence 
25 Table 2, and are embedded one after another. 

Processing by the offset matching unit 2003 in 
this embodiment will be described below, referring to 
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FIG. 28. FIG. 28 is a flowchart for illustrating 
processing by the offset matching unit 2003 in this 
embodiment. The following explanation will be 
presented in accordance with the flowchart in FIG. 28. 
5 In the offset matching unit 2003, for the inputted 

image data wl 2 f , the assumption is made that the 
rotation angle (|) of the pattern arrangement equals 0, 
and at the same time the maximum value MAX is set at 0, 
in Step 2808. 

10 Then, in accordance with the rotation angle <J) 

determined in Step 2809, rotation is applied to the 
coordinate where start bits embedded thereat to 
determine the position of start bits for the rotation 
angle <J>. Then, it is assumed that the coordinate at 

15 the upper-left corner is a start-of -embedment 

coordinate, in Step 2801. At the same time, the 
maximum value MAX is defined set to 0. Then, by Step 
2802, detection of start bits is tried using the 
confidence distance computing unit in FIG. 6. 

20 Whether or not first to fifth bit information 

obtained here is correct start bits "11111" is 
determined by Step 2803. If this point is a correct 
coordinate for starting embedment, five continuous 
positive confidence distances d are detected as 

25 detection results, but if riot so, there are many cases 
where those five positive confidence distances d are 
not continuous. The above described determination may 
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be performed in succession to determine as a coordinate 
for starting embedment the position allowing correct 
start bits Infi to be detected. 

Actually, however, there may be cases where 
5 correct start bits Infi are detected even with a point 
other than a coordinate for starting embedment. The 
cause for it will be described using FIGS. 27A to C. 

FIGS. 21 A to C show the situation in which for 
extracting added information Inf embedded through the 

10 patchwork method for use in this embodiment, the 

pattern arrangement (2702, 2705) (See also the cone 
mask as placement information) identical to the pattern 
arrangement used at the time of embedment of added 
information Inf is used to perform convolution while 

15 making a search for the positions of original mask 

blocks (2701, 2703, 2704). Searches are continuously 
made from the left figure toward the right figure. 

In FIGS. 27A to C, attention is given to one macro 
block (a minimum unit allowing added information Inf to 

20 be extracted) being part of image data wl 2 ', for 

simplification. One square in this figure shows a 
concept of the size of the pattern arrangement for 
embedding 1 bit information. 

If 2701 and 2702 have relationship with each other, 

25 in FIG. 27A, that is if the macro block 2702 is 

positioned at the upper-left compared to the actual 
macro block 2701, the positions of the original image 
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and the pattern arrangement for embedding added 
information Inf overlap one another only in the shaded 
area. 

Also, in FIG. 27B, the case where searches are 
5 further made, and the position, for which searches are 
being made, perfectly matches the position of the 
actual macro block is shown. In this status, the 
pattern arrangement targeted for convolution and the 
£3 macro block overlap one another in a maximum area. 

«a 10 Furthermore, in FIG. 27C, the position, for which 

ri I 

s p searches are being made, is placed at the lower right 

z compared to the position of the macro block in which 

^ l added information Inf is actually embedded. In this 

s 

^ status, the pattern arrangement targeted for 

O 15 convolution and the macro block overlap one another 

Q only in the shaded area. 

In all cases of FIGS. 27A to C, if the pattern 
arrangement targeted for convolution and the macro 
block adequately overlap one another, correct start 
20 bits Infi can be extracted. However, for these three 
cases, confidence distances d are different because 
areas of overlap are different. 

Consideration can be made with the above described 
area of overlap being in place of the aforesaid 
25 confidence distance d. Namely, if the pattern 

arrangement targeted for convolution and the macro 
block perfectly match each other in terms of positional 
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relationship, the confidence distance d is very close 
to ±32c 2 described above, for each bit information. 

Thus in this embodiment, as in the case of FIG. 28, 
if it is determined in Step 2803 that start bits Infi 
5 are not correct ones, movement to a next search point 
is made by Step 2804 according to the raster order. 

On the other hand, it is determined that start 
bits Infi are correct ones, whether or not the sum of 
confidence distances d corresponding to five bits 

10 regarded as start bits Infi is larger than the maximum 
value MAX is determined by Step 2804. 

If the sum is smaller than the maximum value MAX, 
movement to a next search point is made by Step 2807 
according to the raster order. On the other hand, if 

15 the sum of confidence distances d corresponding to five 
bits regarded as start bits Infi is larger than the 
maximum value MAX, the maximum value MAX is updated, 
and the current search point is stored as a starting 
point at the same time. Then, in Step 2806, whether or 

20 not searches have been made for all the search points 
is determined. 

If it is determined in Step 2806 that searches 
have not been made yet for all the search points, 
movement to a next search point is made by Step 2807 

25 according to the raster order. On the other hand, 

advancement to Step 2810 is made to determine whether 
or not all the search angles have been searched if 
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wholly completed, and rotation angles <|> are increased 
to continue to make a search according to Step 2811 if 
not wholly completed. Furthermore, in Step 2811, if 
quick search is made, it is not necessary to make a 
5 search for all angles, but only angles or those in the 
vicinity thereof at which impulse signals stand in 
amplitude spectra in registration processing may be 
searched . 

In Step 2810, if it is determined that a search 
10 has been made for all the search angles, the start-of- 
embedment point and rotation angle (|> stored at that 
time are outputted to end processing. 

Through the processing described above, the offset 
15 matching unit 2003 in this embodiment detects the start 
bits Infi, determines as a point for starting embedment 
of added information Inf the information of the 
coordinate and rotation angle (|> for which the sum of 
confidence distances d corresponding to five bits 
20 regarded as start bits Infi is the largest in 

coordinates for which correct start bits Infi have been 
obtained, and outputs the information to the subsequent 
stage as the coordinate for starting embedment and 
rotation angle. 
25 [3-7-5. Usage information extracting unit] 

The usage information extracting unit 2004 inputs 
the start-of -embedment coordinate, the rotation angle (|> 
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and image data with added information Inf embedded 
therein from the offset matching unit 2003 in the 
previous stage. Furthermore, at this time, the 
position for extraction of usage information should be 
5 rotated in accordance with the rotation angle <{), as in 
the case of the above described offset matching 
processing . 

Hereinafter, for simplifying illustration, in the 
Cj * case where rotation of rotation angle (|> is applied from 

CO 10 the original image to the image data with added 

£ information Inf embedded therein, rotation is applied 

j= in the position for extraction of usage information in 

the usage information extracting unit and position for 
"~ extraction in extraction processing according to the 

ssjss 

^ 15 second pattern arrangement described in detail later, 

W in accordance with the rotation angle <j). 

The usage information extracting unit 2004 
calculates the confidence distance d only for each bit 
information constituting usage information Inf 2 in this 
20 case using in a same way the operations described with 
FIG. 6, and outputs the confidence distance dl for 
these bit information to a statistics inspecting unit 
2006 in the subsequent stage. 

Furthermore, obtaining the confidence distance dl 
25 corresponding to each bit information constituting 

usage information Inf2 substantially means extracting 
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each bit of embedded usage information Inf 2 . 
Description thereof will be presented later. 

Here, each confidence distance d is only 
calculated on the basis of the coordinate for starting 
5 embedment for which determination has been made through 
the above described search, and five bits of start bits 
Infi are not extracted. 

[3-8. Statistics inspection processing] 
The statistics inspecting unit 2006 determines the 
10 reliability of the confidence distance dl obtained by 
the usage information extracting unit 2004 in FIG. 20. 
This determination is carried out by generating the 
confidence distance d2 using a second pattern 
arrangement different from the first pattern 
15 arrangement used for embedding added information Inf 
(usage information Inf 2 ) , and generating a confidence 
index D referring to the occurrence frequency 
distribution of this confidence distance d2 . 

Here, the confidence distance dl is a confidence 
20 distance that is obtained by using the first pattern 
arrangement (See also the cone mask as placement 
information) to extract usage information Inf 2 in the 
usage information extracting unit 2004, and the second 
confidence distance d2 is a confidence distance that is 
25 obtained by using the second pattern arrangement 

described later, which is different form the first 
pattern arrangement . 
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The first pattern arrangement is usually the 
pattern arrangement in FIG. 9 used at the time of 
embedding added information Inf (start bits Infi, usage 
information Inf2) . Furthermore, details about the 
5 second pattern arrangement, the confidence index D and 
the like will be described later. 

[3-8-1. Extraction processing with the second 
pattern arrangement ] 

<<Central limit theorem>> 
10 Subsets A and B are sets constituted by N elements 

expressed by A = {ai, a2, . . . a n } and B = {bi, b2, . . . 
b n }, respectively, each of which is a pixel value held 
by each element of subsets A and B as shown in FIG. 30. 
If for the confidence distance d, i.e. (X(ai - 
15 bi)/N), N takes on an adequately large value, and there 
is no correlation between ai and b±, the expected value 
of the confidence distance d equals 0. The 
distribution of the confidence distance d takes on an 
independent normal distribution by the central limit 
20 theorem. 

Now, the central limit theorem will be briefly 
described. It is a theorem stating that when an 
optional sampling with the size of n c is extracted from 
a population with the average value of mc and the 
25 standard deviation of a c (not necessarily a normal 

distribution) , the distribution of the sampling average 
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value S c approaches a normal distribution 
N (m c , (CTc/'Vnc) ^2 ) as n c increases. 

Generally the standard deviation a c of the 
population is often unknown, but it is almost all right, 
in practice, to use the standard deviation s c of the 
sampling in place of a c when the number of samples n c 
is adequately large and further the number of 
populations N c is large compared to the number of 
samples n c . 

Explanation is now returned to this embodiment. 
First, the occurrence frequency distribution of the 
confidence distance dl determined by the usage 
information extracting unit 2004 is largely varied 
depending on whether or not usage information Inf2 
could be correctly extracted. 

For example, if start bits Infi are erroneously 
detected (if offset matching is failed, and so on) , the 
occurrence frequency distribution of the confidence 
distance dl takes on a normal distribution similar to 
the normal distribution 2501 shown in FIG. 25 because 
bit information is not actually embedded in the 
position in which usage information Inf2 should have 
been embedded. 

On the other hand, if start bits are correctly 
extracted, each confidence distance dl corresponding to 
bit information "1" constituting usage information Inf 2 
is accumulated one after another in the position of the 
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normal distribution 2502, and each confidence distance 
dl corresponding to bit information "0" constituting 
usage information Inf2 is accumulated one after another 
in the position of the normal distribution 2503. Hence, 
5 two "peaks" appear in this case. The ratio between 
sizes of these two "peaks" almost equals the ratio 
between bit information "1" and "0" constituting usage 
information Inf2. 
O However, this is based on the premise that the 

10 confidence distance dl obtained by performing 

convolution processing according to the first pattern 
_?2 arrangement for the original image in which added 

h j : information is not embedded takes on a normal 

distribution similar to the normal distribution 2501. 
4f 15 Therefore, in actuality, whether or not extraction 

p has been correctly performed can not be determined 

unless the state of the original image is known. 

Thus, in this embodiment, the normal distribution 
of the confidence distance d2 is generated using so 
20 called a second pattern arrangement allowing the state 
of the image to be determined adequately even though 
added information is embedded, and this normal 
distribution is considered as the normal distribution 
2501, thereby determining whether or not usage 
25 information Inf 2 has been correctly extracted. 

For example, if the occurrence frequency 
distribution of the confidence distance dl exists 
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outside a shaded area (constituent elements occupying 
the range from the center to the point of 95 %) 
constituting the normal distribution 2501 generated 
with the confidence distance d2, it can be considered 
5 that a statistical deviation is caused in the targeted 
image and usage information Inf2 is embedded therein, 
thus making it possible to determine the reliability of 
the usage information Inf2 statistically. A detailed 
method thereof will be described later. 

10 A method in which an occurrence frequency 

distribution similar to that of the confidence distance 
dl before embedment of added information Inf (similar 
to the normal distribution 2501 shown in FIG. 25) is 
generated, using image data in which added information 

15 Inf (usage information Inf2) is embedded, will be now 
described . 

In this embodiment, the extracting unit 2005 
according to the second pattern arrangement is used to 
determine the confidence distance d2 constituting a 

20 distribution similar to the normal distribution 2501. 

The extracting unit 2005 is a unit that determines 
the confidence distance d2 using the second pattern 
arrangement "orthogonal to" the first pattern 
arrangement used with the usage information extracting 

25 unit 2004, and is almost identical to the usage 

information extracting unit 2004 in terms of operation 
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itself, in that it performs convolution processing, and 
so on . 

Furthermore, for the purpose of contrastive 
illustration, the pattern arrangement in FIG. 9 used 
5 with the usage information extracting unit 2004 and the 
mask (cone mask) for referring to the position in which 
this pattern arrangement is placed are called a "first 
pattern arrangement" and a " first position reference 
£3 mask", respectively, and the pattern arrangement 

£0 10 "orthogonal to" the first pattern arrangement and the 

ru 

mask for referring to the position in which this 
_g pattern arrangement is placed are called a "second 

" r * pattern arrangement" and a "second position reference 

' S Z mask", respectively. 

15 A coordinate for starting embedment is first 

<3 inputted in the extracting unit 2005 according to the 

second pattern arrangement from the offset matching 
unit 2003, and the confidence distance d2 is calculated 
using confidence distance computation in FIG. 6 
20 described above, as well. 

At this time, the pattern arrangement that is used 
in confidence distance computation in FIG. 6 is not the 
pattern arrangement in FIG. 9 used in embedment 0901 
but a pattern arrangement 3301 or 3302 "orthogonal to" 
25 this pattern arrangement 0901 as shown in FIGS. 33A and 
B. 
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This is because the confidence distance d2 that is 
calculated using the pattern arrangements 3301 and 3302 
in FIGS. 33A and B is not affected at all by operations 
with the pattern arrangement 0901 in FIG. 9. 
5 As shown in FIG. 34, as a result of performing 

processing of convolution of the pattern arrangement 
0901 in FIG. 9 and the above described pattern 
arrangement 3301 "orthogonal" thereto, 0 is obtained. 
^0 The same goes for the pattern arrangement 3302. In 

fy 10 other words, as a result of convolution of the first 

H -J and second pattern arrangements, 0 is obtained. Thus, 

ol even though the gray scale of the original image is 

q changed using the first pattern arrangement, the 

confidence distance d obtained by performing 
15 convolution processing using the second pattern 
arrangement is not affected at all. 

Therefore, the occurrence frequency distribution 
of the confidence distance d2 obtained by performing 
convolution processing using the above described second 
20 pattern arrangement for the image in which added 
information Inf is embedded is almost same as the 
normal distribution 2501 in FIG. 25. Accordingly, the 
occurrence frequency distribution is regarded as the 
normal distribution 2501. 
25 The normal distribution 2501 obtained at this time 

provides a criterion of determination required for 
statistics inspection processing of 3207 in FIG. 32. 
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In extraction processing 2005 according to the 
second pattern arrangement, the normal distribution of 
the confidence distance d2 is generated, using "pattern 
arrangement "orthogonal to" the first pattern" as shown 
5 by 3301 and 3302 in FIGS. 33 A and B and the second 
position reference mask shown by 3502 in FIG. 35B. 

Furthermore, the conditions of the above described 
"pattern arrangement "orthogonal to" the first pattern" 
O are as follows: (1) it has the same size as that of 

CQ 10 0901 in FIG. 9 as shown in FIGS. 33A and B, and (2) as 

a result of processing of convolution with the pattern 
_? arrangement in FIG. 9 used at the time of embedding 

l ?i added information Inf, 0 is obtained as in the case of 

the pattern arrangements 3301 and 3302. 
y 15 Also, convolution processing shown in FIG. 34 is 

P same as convolution processing shown in FIGS. 21 and 22. 

In this embodiment, the case where 0 is obtained 
as a result of convolution is called a case where 
"pattern arrangements are "orthogonal to " each other", 
20 linking to the case where 0 is obtained when inner 
products of vectors are orthogonal with each other. 
Thus, 3301 and 3302 in FIGS. 33A and B represent 
"pattern arrangements "orthogonal to" the pattern 
arrangement 0901 in FIG. 9". 
25 The reason for using the pattern arrangement 

"orthogonal to" the pattern arrangement used in 
embedment of added information to calculate the 
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confidence distance d2 is to prevent a statistical 
deviation being caused in the distribution of the 
confidence distance d2 , namely, to generate an 
occurrence of frequency distribution centered on 0. 

Also, as a necessary condition of the "pattern 
arrangement "orthogonal to" the first pattern", (3) it 
has the same number of non-zero elements as non-zero 
elements of the pattern arrangement used for usage 
information extraction processing 2004, and the numbers 
of positive and negative elements are the same, 
respectively. This is necessary to extract the 
confidence distances dl and d2 under the same 
computation condition. Then in this embodiment, for 

the "second position reference mask", the mask shown by 
3502 in FIG. 35B that has a pattern different from that 
of 3501 in FIG. 35A used at the time of embedding added 
information Inf and is different in size from the mask 
3501 is used. 

As described above, if the above described first 
and second pattern arrangements are different from each 
other, the occurrence frequency distribution of the 
confidence distance d2 is almost same as the normal 
distribution 2501 . 

However, there is also a possibility that a 
statistical deviation is detected in spite of the fact 
that the second pattern arrangement has been used to 
perform convolution, if detecting start bits position 
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is not perfect, and so on. In this embodiment, the 
sizes of the first and second position reference masks 
are made to be different from each other, considering 
this possibility, whereby periodic elements are 
5 canceled out. Alternatively, each pattern arrangement 
in the mask is placed in a different way, thereby 
preventing convolution in the same area. 

Also, in this case, the "second position reference 
mask" only needs to have each coefficient constituting 
10 the mask distributed at random, and is not necessarily 
a cone mask. 

If a setting is made so that the "second embedment 
position reference mask" is different from the "first 
embedment position reference mask", the "second 

15 embedment position reference mask" shall be created by 

the embedment position determining unit 2008 in FIG. 20. 

Generally, considering the above described 
resistance to clipping, it is unlikely that the first 
position reference mask (cone mask) does not take on a 

20 large size relative to the entire image data targeted 
for embedment of added information. Thus, for the 
"second position reference mask", it is advisable to 
use a larger mask. In this embodiment, a setting is 
made so that the second mask that is used at the time 

25 of calculating the confidence distance dl at the added 
information Inf side is larger in size that the first 
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mask to which reference is made at the time of 
embedding added information Inf. 

However, this embodiment is limited thereto, and 
even if the sizes of the both masks are the same, some 
5 degree of effect is provided. Therefore, the "second 
position reference mask" may be a mask created by the 
position determining unit 2002 in FIG. 20. 

For a minimum condition of the mutual masks, it is 
necessary that the numbers of repetitions of respective 
10 bits constituting added information applied to the 

mutual masks are the same in image areas having same 
sizes . 

Furthermore, if satisfactory results cannot be 
obtained with extraction processing according to the 

15 second pattern arrangement, there is also a possibility 
that an ideal occurrence frequency distribution that is 
2501 in FIG. 25 can be generated by calculating the 
confidence distance d2 again using another pattern 
arrangement and second position reference mask 

20 satisfying the above described conditions. 

Specific operations of the extracting unit 2005 
according to the second pattern arrangement will be now 
presented . 

In this embodiment, the first position reference 
25 mask is a 32x32 cone mask and the second position 

reference mask is a 64x64 cone mask, and the relative 
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arrangement of each coefficient is entirely different 
for these two masks. 

First, for the second pattern arrangement 2005, 
determination of extraction positions shall be made in 
accordance with the following Correspondence Table 3. 

[Correspondence Table 3] 



The order of each bit 
information 



69 



^0 

£0 



Coefficient values in the 
second position reference mask 



0,1 



2,3 



4,5 



6,7 



136,137 



For the second position reference mask, there 
exist sixteen coefficients having the same values 

10 respectively in the mask. For the first position 

reference mask of 32x32, on the other hand, the number 
of repetitions of the same coefficient among 32x32 
coefficients is 4, in the case where reference to the 
mask is made using the previous Correspondence Table 2. 

15 That is, for both the first and second position 

reference masks, coefficients having same values exist 
in equal numbers. 

In this embodiment, the second pattern arrangement 
is assigned to the positional relation conforming to 

20 the rule of the above described Correspondence Table 3, 



- 133 - 



and convolution processing is performed, one after 
another, to calculate 69 confidence distances d2 
corresponding to respective bit information. 
[3-8-2. Confidence index D] 
5 The confidence distance d2 generated by the 

extracting unit 2005 according to the second pattern 
arrangement 2005 occurs in distribution almost the same 
as the normal distribution 2501, but in the normal 
SSL distribution, it is generally known that 95 % of 

10 samples (confidence distances d2) in the range defined 
^ by the following equation (25.1) occur. 

Nj m-l . 96a<d2<m+l . 96a ... Equation (25.1) 

01 wherein a is a standard deviation for the above 

p described confidence distance d2, and m is an average 

rk 15 value. Furthermore, the range in the above described 

IS case is called a " 95 % confidence interval". 

? After the confidence distance d2 is obtained in 

the extraction unit 2005 according to the second 
pattern arrangement, m - 196a and m + 1.96a are 
20 calculated using the confidence distance d2 . 

Since the occurrence frequency distribution of the 
confidence distance dl inputted in the statistics 
inspecting unit 2006 from the usage information 
extracting unit 2004 takes on the normal distribution 
25 2502 in FIG. 25 in the case of bit information of 1", 
and takes on the normal distribution 2503 in the case 
of bit information of "0" , the possibility is quite 
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high that the confidence distance dl corresponding to 
usage information Inf 2 exists outside the 95 % 
confidence interval (a shaded area in FIG . 25) obtained 
by the extracting unit 2005 according to the second 
5 pattern arrangement. 

By the way, if at the time of processing by the 
offset matching unit 2003, no added information Inf 2 
exists in the image targeted for this processing, the 
occurrence frequency distribution of the confidence 

10 distance dl also takes on a distribution similar to the 
normal distribution 2501. 

The possibility that none of 64 confidence 
distances dl corresponding to usage information Inf 2 is 
included in the confidence interval of the equation 

15 (25.1) in spite of the fact that added information Inf 2 
is embedded in the image is (1-0.95) raised to the 64th 
power, which is extremely low. 

Therefore, as long as the normal distribution 2501 
has been determined based on the confidence distance d2, 

20 whether or not the occurrence frequency distribution 
determined based on the confidence distance dl is 
included in the range occupying most part of this 
normal distribution is considered, thereby making it 
possible to determine almost reliably whether or not 

25 added information Inf (usage information Inf 2 ) is 
embedded . 
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The statistics inspecting unit 2006 determines a 
confidence degree as to whether added information 
(usage information Inf2) is embedded, using properties 
as described above. 
5 In this embodiment, the confidence degree as to 

whether usage information Inf is embedded is considered 
as a confidence index D. 

This confidence index D is defined as the ratio of 
the number of confidence distances dl existing outside 

10 the range of the equation (25.1) to the number of all 
the confidence distances dl generated by the usage 
information extracting unit 2004. 

If this confidence index D is larger than a 
threshold value Th, the statistics inspecting unit 2006 

15 determines that the general occurrence frequency 
distribution of the confidence distance dl is 
artificially biased so that confidence distances dl are 
distributed heavily in positions like 2502 and 2503, 
namely it is an image in which usage information Inf2 

20 is surely embedded. Thus, it is considered that the 
confidence distance dl used for determination is in 
itself reliable information, and it is permitted that 
this confidence distance dl is further transferred to a 
comparing unit 2007 in the subsequent stage. 

25 Furthermore, for the confidence index D, the 

confidence index D of usage information Inf 2 or a 
massage based on the index D may be displayed on a 
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monitor, as shown by a confidence displaying step 3210 
in FIG. 32. 

For example, if the confidence index D is not 
larger than the threshold value Th, a massage like 
"Usage information Inf2 is not correctly extracted", 
followed by returning from a statistics inspecting step 
3207 to a step of inputting an image again 3202 in FIG. 
32. 

[3-9. Comparison processing] 

The comparing unit 2007 in FIG. 20 inputs the 
confidence distance dl outputted via the usage 
information extracting unit 2004 and the statistics 
inspecting unit 2006. Because the confidence distance 
dl that is inputted therein is highly reliable 
information, all that should be done here is to simply 
determine whether respective bit information 
corresponding to confidence distances dl is of "1" or 
"0" . 

Specifically, if the confidence distance dl of 
certain bit information constituting usage information 
Inf 2 is positive, then it is determined that this bit 
information is of "1", and if the confidence distance 
dl is negative, then it is determined that this bit 
information is of "0". 

The usage information Inf 2 obtained through the 
above described determination is outputted as final 



data for providing user's reference information or the 
control signal. 

Explanation of a series of processing from 
embedment of added information to extraction thereof 
5 has been presented above. 

<Other embodiment s> 

For added information Inf (usage information lnf 2 ) 
in one embodiment described above, error correction- 
coded information can also be used. Thus, error 
10 correction-coded information is used for added 

£0 

Hi information Inf (usage information Inf2) , thereby 

r '-4 further enhancing reliability of usage information Inf 2 

Cfi already extracted. 

q Also, in the first embodiment described above, the 

15 case where a pattern arrangement most suitable for the 
resolution of input image data or the output (print) 
E "~ resolution of the printer is selected from two pattern 

arrangements shown in FIGS. 49A and B, but the present 
invention is not limited the above described example, 
20 and for example, the case where two or more pattern 
arrangements different in size from one another, 
constituted by mxn elements (m, n are integer numbers) 
are held and one pattern arrangement most suitable for 
the resolution of input image data or the output 
25 (print) resolution of the printer is selected therefrom 
and is used is also included in the category of the 
present invention . 
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Furthermore, the present invention may be applied 
as part of a system constituted by a plurality of 
apparatuses (for example, host computer, interface 
device, printer, etc.) or may be applied as part of a 
system constituted by one apparatus (for example, 
copier, facsimile machine) . 

Also, the present invention is not limited to 
devices and methods for achieving the above embodiments, 
and the case is also included in the category of the 
present invention where the program code of software 
for achieving the above described embodiment is 
provided to the computer in the above described system 
or apparatus (CPU or MPU) , and the computer in the 
above described system or apparatus operates a various 
kinds of devices described above, there by achieving 
the above described embodiment. 

Also, in this case, the program code of the above 
described software itself achieves the feature of the 
above described embodiment, and the program code itself 
and a device for supplying the program code to the 
computer, specifically a storage medium storing the 
above described program code is included in the 
category of the present invention. 

As for a storage medium like this, for example a 
floppy disk, hard disk, optical disk, magneto-optic 
disk, CD-ROM, magnetic tape, nonvolatile memory and ROM 
may be used. 



Also, the program code is included in not only the 
case where the above described computer controls a 
various kinds of devices in accordance with only the 
supplied program code, thereby achieving the feature of 
5 the above described embodiment, but also the case where 
the above described program code works in association 
with the OS (operating system) operating on the 
computer, other application software or the like to 
m achieve the above described embodiment. 

~- y 10 Furthermore, the case is also included in the 

^j=j category of the present invention where after this 

N supplied program code is stored in a memory included in 

£m the feature expansion board of the computer and the 

□ feature expansion unit connected to the computer, a CPU 

rjg 15 and the like included in the feature expansion board 

?2 and the feature storage unit performs all or part of 

practical processing based on instructions of the 
program code, and the above described embodiment is 
achieved through the processing. 
20 Furthermore, for the above described embodiment, 

the case where digital watermark information is 
embedded using the cone mask is described, but the 
present invention is not limited thereto. Particularly, 
cases where digital watermark information is embedded 
25 using the blue noise mask are also included in the 
category of the present invention. 
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Also, any configuration comprising at least one of 
variety of characteristics described above is included 
in the category of the present invention. 

As described above, according to this embodiment, 
in registration processing of correcting geometric 
transformation, it is made easy to detect the 
registration signal and the registration signal can be 
embedded so that it is less perceptible by human eyes, 
compared to conventional cases. 

As many apparently widely different embodiments of 
the present invention can be made without departing 
from the sprit and scope thereof, it is to be 
understood that the invention is not limited to the 
specific embodiments thereof except as defined in the 
appended claims. 



